2024-04-28T12:19:23.427 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-28T12:19:23.427 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} --archive /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882 --name rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --verbose -- /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/orig.config.yaml 2024-04-28T12:19:23.484 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-28T12:19:23.579 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882 branch: wip-rishabh-testing-20240426.111959 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} email: null first_in_suite: false job_id: '7676882' kernel: client: branch: testing flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEIkHMXbbD8+hFeko3auXYSTn1U+R4Jvg6s6D1VvWpNpIaWZ3ipFyVXRkbetjB3dx0/6Nz3XztJamto3A3rHySY= smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDsPJ7LPnuQZtmVlRbuR5Cq3lxEvksEO95xJc/qlvWrk3lVwrU5/15o0OjhOw+izl6mc2fkmYTXh0gEWCGcnDLE= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKkE+47+lLqX/2/2xhe2CM41Z7iwkeNOx2ZAE8YxafIUhVkPqow7M0DqIbZcEkzbI1Ze0Kis5IN6VdlHuclbd+0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-28_11:41:23 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-28T12:19:23.579 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-28T12:19:23.580 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-28T12:19:23.580 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-28T12:19:23.581 INFO:teuthology.task.internal:Checking packages... 2024-04-28T12:19:23.604 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-28T12:19:23.604 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-28T12:19:23.604 INFO:teuthology.packaging:ref: None 2024-04-28T12:19:23.605 INFO:teuthology.packaging:tag: None 2024-04-28T12:19:23.605 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-28T12:19:23.605 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:19:23.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-28T12:19:23.742 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-28T12:19:23.743 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-28T12:19:23.749 INFO:teuthology.task.internal:no buildpackages task found 2024-04-28T12:19:23.749 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-28T12:19:23.794 INFO:teuthology.task.internal:Saving configuration 2024-04-28T12:19:23.814 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-28T12:19:23.820 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T12:19:23.839 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:16.228162', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEIkHMXbbD8+hFeko3auXYSTn1U+R4Jvg6s6D1VvWpNpIaWZ3ipFyVXRkbetjB3dx0/6Nz3XztJamto3A3rHySY='} 2024-04-28T12:19:23.855 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:16.226939', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDsPJ7LPnuQZtmVlRbuR5Cq3lxEvksEO95xJc/qlvWrk3lVwrU5/15o0OjhOw+izl6mc2fkmYTXh0gEWCGcnDLE='} 2024-04-28T12:19:23.873 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:16.229271', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKkE+47+lLqX/2/2xhe2CM41Z7iwkeNOx2ZAE8YxafIUhVkPqow7M0DqIbZcEkzbI1Ze0Kis5IN6VdlHuclbd+0='} 2024-04-28T12:19:23.873 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-28T12:19:23.879 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-28T12:19:23.879 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-28T12:19:23.879 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-28T12:19:23.879 INFO:teuthology.run_tasks:Running task console_log... 2024-04-28T12:19:23.982 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3a70c27160>, signals=[15]) 2024-04-28T12:19:23.983 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-28T12:19:23.989 INFO:teuthology.task.internal:Opening connections... 2024-04-28T12:19:23.989 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-04-28T12:19:23.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:19:24.066 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-04-28T12:19:24.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:19:24.141 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-04-28T12:19:24.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:19:24.214 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-28T12:19:24.220 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-04-28T12:19:24.239 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-04-28T12:19:24.239 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-28T12:19:24.296 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-04-28T12:19:24.297 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-28T12:19:24.297 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-28T12:19:24.297 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-28T12:19:24.297 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-04-28T12:19:24.313 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-04-28T12:19:24.331 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-04-28T12:19:24.331 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-04-28T12:19:24.388 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-04-28T12:19:24.389 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-04-28T12:19:24.389 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-04-28T12:19:24.389 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-04-28T12:19:24.389 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-04-28T12:19:24.389 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-04-28T12:19:24.389 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-28T12:19:24.390 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-04-28T12:19:24.390 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-04-28T12:19:24.390 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-28T12:19:24.390 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-04-28T12:19:24.390 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-28T12:19:24.390 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-28T12:19:24.390 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-28T12:19:24.390 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-04-28T12:19:24.405 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-04-28T12:19:24.423 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-04-28T12:19:24.423 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-04-28T12:19:24.480 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-04-28T12:19:24.480 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-04-28T12:19:24.480 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-04-28T12:19:24.480 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-04-28T12:19:24.480 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-04-28T12:19:24.481 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-04-28T12:19:24.481 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-28T12:19:24.481 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-04-28T12:19:24.481 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-04-28T12:19:24.481 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-28T12:19:24.481 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-04-28T12:19:24.481 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-28T12:19:24.481 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-28T12:19:24.481 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-28T12:19:24.481 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-04-28T12:19:24.538 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-28T12:19:24.548 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-28T12:19:24.561 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-28T12:19:24.561 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-04-28T12:19:24.564 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-04-28T12:19:24.566 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-04-28T12:19:24.583 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-28T12:19:24.588 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-28T12:19:24.588 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-04-28T12:19:24.621 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-04-28T12:19:24.623 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-04-28T12:19:24.791 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-28T12:19:24.852 INFO:teuthology.run_tasks:Running task kernel... 2024-04-28T12:19:24.868 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-28T12:19:24.869 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-28T12:19:24.869 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-28T12:19:24.869 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-28T12:19:24.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:19:24.886 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-28T12:19:25.162 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:25.162 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-28T12:19:25.162 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-04-28T12:19:25.180 INFO:teuthology.orchestra.run.smithi106.stdout:5.14.0-437.el9.x86_64 2024-04-28T12:19:25.180 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:25.180 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-28T12:19:25.180 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum upgrade -y linux-firmware 2024-04-28T12:19:31.020 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.0 MB 00:04 2024-04-28T12:19:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-28T12:19:38.848 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.2 MB 00:02 2024-04-28T12:19:40.606 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-28T12:19:41.799 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-28T12:19:46.932 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-28T12:19:48.914 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-28T12:19:48.915 INFO:teuthology.orchestra.run.smithi106.stdout:Nothing to do. 2024-04-28T12:19:48.915 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-28T12:19:49.034 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-28T12:19:49.035 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-28T12:19:49.178 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-28T12:19:49.178 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-28T12:19:49.181 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi106.front.sepia.ceph.com to complete... 2024-04-28T12:23:44.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28 12:23:44 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-28T12:23:44.668 INFO:teuthology.orchestra.run.smithi106.stderr:FINISHED --2024-04-28 12:23:44-- 2024-04-28T12:23:44.669 INFO:teuthology.orchestra.run.smithi106.stderr:Total wall clock time: 3m 55s 2024-04-28T12:23:44.669 INFO:teuthology.orchestra.run.smithi106.stderr:Downloaded: 1 files, 341M in 3m 55s (1.45 MB/s) 2024-04-28T12:23:44.673 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-28T12:23:44.673 DEBUG:teuthology.orchestra.run.smithi106:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-28T12:23:46.770 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying... ######################################## 2024-04-28T12:23:46.903 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing... ######################################## 2024-04-28T12:23:47.015 INFO:teuthology.orchestra.run.smithi106.stdout:Updating / installing... 2024-04-28T12:23:54.972 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-28T12:24:32.697 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi106.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-28T12:24:32.698 DEBUG:teuthology.orchestra.run.smithi106:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-28T12:24:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-28T12:24:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-28T12:24:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-28T12:24:32.785 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-28T12:24:32.786 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-28T12:24:32.787 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-28T12:24:32.788 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-28T12:24:32.789 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-28T12:24:32.790 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-28T12:24:32.791 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-28T12:24:32.792 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-28T12:24:32.793 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-28T12:24:32.794 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-28T12:24:32.795 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-28T12:24:32.796 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-28T12:24:32.797 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-28T12:24:32.798 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-28T12:24:32.799 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-28T12:24:32.800 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-28T12:24:32.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-28T12:24:32.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-28T12:24:32.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-28T12:24:32.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-28T12:24:32.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-28T12:24:32.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-28T12:24:32.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-28T12:24:32.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-28T12:24:32.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-28T12:24:32.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-28T12:24:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-28T12:24:32.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-28T12:24:32.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-28T12:24:32.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-28T12:24:32.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-28T12:24:32.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-28T12:24:32.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-28T12:24:32.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-28T12:24:32.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-28T12:24:32.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-28T12:24:32.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-28T12:24:32.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-28T12:24:32.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-28T12:24:32.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-28T12:24:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-28T12:24:32.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-28T12:24:32.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-28T12:24:32.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-28T12:24:32.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-28T12:24:32.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-28T12:24:32.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-28T12:24:32.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-28T12:24:32.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-28T12:24:32.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-28T12:24:32.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-28T12:24:32.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-28T12:24:32.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-28T12:24:32.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-28T12:24:32.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-28T12:24:32.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-28T12:24:32.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-28T12:24:32.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-28T12:24:32.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-28T12:24:32.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-28T12:24:32.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-28T12:24:32.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-28T12:24:32.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-28T12:24:32.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-28T12:24:32.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-28T12:24:32.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-28T12:24:32.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-28T12:24:32.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-28T12:24:32.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-28T12:24:32.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-28T12:24:32.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-28T12:24:32.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-28T12:24:32.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-28T12:24:32.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-28T12:24:32.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-28T12:24:32.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-28T12:24:32.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-28T12:24:32.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-28T12:24:32.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-28T12:24:32.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-28T12:24:32.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-28T12:24:32.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-28T12:24:32.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-28T12:24:32.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-28T12:24:32.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-28T12:24:32.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-28T12:24:32.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-28T12:24:32.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-28T12:24:32.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-28T12:24:32.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-28T12:24:32.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-28T12:24:32.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-28T12:24:32.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-28T12:24:32.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-28T12:24:32.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-28T12:24:32.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-28T12:24:32.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-28T12:24:32.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-28T12:24:32.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-28T12:24:32.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-28T12:24:32.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-28T12:24:32.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-28T12:24:32.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-28T12:24:32.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-28T12:24:32.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-28T12:24:32.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-28T12:24:32.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-28T12:24:32.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-28T12:24:32.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-28T12:24:32.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-28T12:24:32.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-28T12:24:32.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-28T12:24:32.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-28T12:24:32.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-28T12:24:32.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-28T12:24:32.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-28T12:24:32.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-28T12:24:32.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-28T12:24:32.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-28T12:24:32.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-28T12:24:32.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-28T12:24:32.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-28T12:24:32.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-28T12:24:32.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-28T12:24:32.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-28T12:24:32.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-28T12:24:32.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-28T12:24:32.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-28T12:24:32.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-28T12:24:32.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-28T12:24:32.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-28T12:24:32.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-28T12:24:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-28T12:24:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-28T12:24:32.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-28T12:24:32.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-28T12:24:32.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-28T12:24:32.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-28T12:24:32.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-28T12:24:32.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-28T12:24:32.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T12:24:32.922 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:32.922 DEBUG:teuthology.orchestra.run.smithi106:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-28T12:24:32.947 INFO:teuthology.orchestra.run.smithi106.stdout:postinstall scriptlet (using /bin/sh): 2024-04-28T12:24:32.947 INFO:teuthology.orchestra.run.smithi106.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T12:24:32.947 INFO:teuthology.orchestra.run.smithi106.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-28T12:24:32.947 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2024-04-28T12:24:32.947 INFO:teuthology.orchestra.run.smithi106.stdout:for file in vmlinuz System.map config; do 2024-04-28T12:24:32.947 INFO:teuthology.orchestra.run.smithi106.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-28T12:24:32.947 INFO:teuthology.orchestra.run.smithi106.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout: fi 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout:done 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:32.948 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2024-04-28T12:24:32.949 DEBUG:teuthology.orchestra.run.smithi106:> sudo rpm -qi grub2-tools 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Name : grub2-tools 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Epoch : 1 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Version : 2.06 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Release : 68.el9 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: x86_64 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Group : Unspecified 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Size : 8264254 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:License : GPLv3+ 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-28T12:24:33.024 INFO:teuthology.orchestra.run.smithi106.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:Packager : builder@centos.org 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:Vendor : CentOS 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:Summary : Support tools for GRUB. 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:Description : 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:hardware devices. 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:24:33.025 INFO:teuthology.orchestra.run.smithi106.stdout:This subpackage provides tools for support of all platforms. 2024-04-28T12:24:33.026 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-28T12:24:33.026 INFO:teuthology.task.kernel:Updating grub on smithi106 to boot 6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:33.027 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-28T12:24:33.588 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2024-04-28T12:24:34.996 INFO:teuthology.orchestra.run.smithi106.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-28T12:24:35.027 INFO:teuthology.orchestra.run.smithi106.stderr:done 2024-04-28T12:24:35.029 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2024-04-28T12:24:35.054 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.6HBds3hVKX 2024-04-28T12:24:35.054 DEBUG:teuthology.orchestra.run.smithi106:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6HBds3hVKX 2024-04-28T12:24:35.119 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.6HBds3hVKX 2024-04-28T12:24:35.251 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.6HBds3hVKX is 6KB 2024-04-28T12:24:35.300 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.6HBds3hVKX 2024-04-28T12:24:35.316 DEBUG:teuthology.orchestra.run.smithi106:> sudo /bin/ls /boot/loader/entries || true 2024-04-28T12:24:35.376 INFO:teuthology.orchestra.run.smithi106.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-28T12:24:35.377 INFO:teuthology.orchestra.run.smithi106.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-28T12:24:35.377 INFO:teuthology.orchestra.run.smithi106.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-28T12:24:35.377 INFO:teuthology.orchestra.run.smithi106.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-28T12:24:35.378 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-28T12:24:35.547 DEBUG:teuthology.orchestra.run.smithi106:> sudo shutdown -r now 2024-04-28T12:25:05.549 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-28T12:25:05.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-28T12:25:05.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:26:05.554 DEBUG:teuthology.orchestra.remote:timed out 2024-04-28T12:26:14.555 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-28T12:26:14.556 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:26:15.939 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-28T12:26:16.644 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-28T12:26:16.644 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-28T12:26:16.644 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2024-04-28T12:26:16.684 INFO:teuthology.orchestra.run.smithi106.stdout:6.9.0-rc3-g806ec36b2112 2024-04-28T12:26:16.684 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-28T12:26:16.684 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-28T12:26:16.684 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-28T12:26:17.688 DEBUG:teuthology.parallel:result is None 2024-04-28T12:26:17.688 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-28T12:26:17.696 INFO:teuthology.task.internal:Creating test directory... 2024-04-28T12:26:17.696 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-28T12:26:17.704 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-28T12:26:17.707 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-28T12:26:17.753 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-28T12:26:17.760 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-28T12:26:17.821 INFO:teuthology.task.internal:Creating archive directory... 2024-04-28T12:26:17.821 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-28T12:26:17.829 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-28T12:26:17.831 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-28T12:26:17.941 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-28T12:26:17.947 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-28T12:26:17.948 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-28T12:26:17.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:26:17.964 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-28T12:26:17.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:26:17.981 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-28T12:26:17.988 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-28T12:26:18.007 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-28T12:26:18.064 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:18.075 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:18.077 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:18.091 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:18.097 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:18.146 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-28T12:26:18.150 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-28T12:26:18.158 INFO:teuthology.task.internal:Configuring sudo... 2024-04-28T12:26:18.158 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-28T12:26:18.163 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-28T12:26:18.165 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-28T12:26:18.224 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-28T12:26:18.232 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-28T12:26:18.233 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-28T12:26:18.237 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-28T12:26:18.239 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-28T12:26:18.273 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:18.389 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:18.478 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:18.549 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:18.631 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:26:18.632 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-28T12:26:18.713 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:18.757 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:18.821 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:18.865 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:18.907 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:26:18.907 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-28T12:26:18.973 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:19.017 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-28T12:26:19.065 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:19.149 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-28T12:26:19.190 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:26:19.190 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-28T12:26:19.256 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-04-28T12:26:19.263 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-04-28T12:26:19.266 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-04-28T12:26:19.308 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T12:26:19.336 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T12:26:19.370 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T12:26:19.804 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-28T12:26:19.811 INFO:teuthology.task.internal:Starting timer... 2024-04-28T12:26:19.812 INFO:teuthology.run_tasks:Running task pcp... 2024-04-28T12:26:19.840 INFO:teuthology.run_tasks:Running task selinux... 2024-04-28T12:26:19.856 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-28T12:26:19.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-04-28T12:26:19.980 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-04-28T12:26:19.984 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-04-28T12:26:20.059 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-04-28T12:26:20.060 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-04-28T12:26:20.114 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-04-28T12:26:20.116 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-28T12:26:20.116 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-04-28T12:26:20.166 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-04-28T12:26:20.166 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-04-28T12:26:20.192 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-04-28T12:26:20.192 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-04-28T12:26:20.218 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-04-28T12:26:20.218 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-04-28T12:26:20.219 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T12:26:20.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:26:20.293 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T12:26:20.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:26:20.320 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T12:26:20.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T12:26:20.349 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-28T12:26:20.349 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-04-28T12:26:20.454 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-04-28T12:26:20.479 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-04-28T12:26:20.505 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-28T12:26:20.516 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-28T12:26:20.537 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-28T12:26:20.539 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi106.front.sepia.ceph.com,smithi129.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-28T12:31:25.197 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-04-28T12:31:25.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-28T12:31:25.200 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:31:25.308 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-28T12:31:25.467 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-28T12:31:25.467 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-28T12:31:25.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:31:25.544 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-04-28T12:31:25.627 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-04-28T12:31:25.628 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-28T12:31:25.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-28T12:31:25.701 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-04-28T12:31:25.777 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-04-28T12:31:25.779 INFO:teuthology.run_tasks:Running task clock... 2024-04-28T12:31:25.789 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-28T12:31:25.790 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-28T12:31:25.791 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T12:31:25.798 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-28T12:31:25.798 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T12:31:25.801 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-28T12:31:25.801 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T12:31:25.830 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-28T12:31:25.845 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-28T12:31:25.846 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-28T12:31:25.859 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-28T12:31:25.870 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-04-28T12:31:25.881 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-04-28T12:31:25.883 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-04-28T12:31:25.885 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-28T12:31:25.893 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-04-28T12:31:25.895 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-28T12:31:25.906 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-28T12:31:25.908 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-28T12:31:25.920 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-28T12:31:25.925 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-28T12:31:25.964 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-04-28T12:31:25.967 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T12:31:25.967 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-04-28T12:31:25.968 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:25.968 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:25.968 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:25.968 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:25.975 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-28T12:31:25.979 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T12:31:25.979 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-28T12:31:25.979 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:25.979 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:25.979 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:25.979 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:25.995 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-04-28T12:31:26.027 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-04-28T12:31:26.063 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-28T12:31:26.104 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-28T12:31:26.250 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-28T12:31:26.260 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T12:31:26.260 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-28T12:31:26.260 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:26.260 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:26.260 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:26.260 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T12:31:26.261 INFO:teuthology.run_tasks:Running task install... 2024-04-28T12:31:26.274 DEBUG:teuthology.task.install:project ceph 2024-04-28T12:31:26.274 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-28T12:31:26.274 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-28T12:31:26.274 INFO:teuthology.task.install:Using flavor: default 2024-04-28T12:31:26.282 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-28T12:31:26.283 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-28T12:31:26.283 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-28T12:31:26.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:31:26.285 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-28T12:31:26.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:31:26.287 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-28T12:31:26.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:31:26.436 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-28T12:31:26.436 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-28T12:31:26.442 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-28T12:31:26.442 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-28T12:31:26.513 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-28T12:31:26.513 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-28T12:31:26.866 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-28T12:31:26.866 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:31:26.867 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-28T12:31:26.874 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-28T12:31:26.874 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:31:26.874 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-28T12:31:26.902 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-28T12:31:26.902 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-28T12:31:26.912 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-28T12:31:26.912 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:31:26.912 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-28T12:31:26.942 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-28T12:31:26.943 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-28T12:31:26.956 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-28T12:31:26.957 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-28T12:31:26.970 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-28T12:31:27.012 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-28T12:31:27.044 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-28T12:31:27.082 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-28T12:31:27.088 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-28T12:31:27.107 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-04-28T12:31:27.109 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-04-28T12:31:27.153 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-04-28T12:31:27.154 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-04-28T12:31:27.193 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-28T12:31:27.272 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2024-04-28T12:31:27.276 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2024-04-28T12:31:27.334 INFO:teuthology.orchestra.run.smithi171.stdout:48 files removed 2024-04-28T12:31:27.362 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-28T12:31:27.383 INFO:teuthology.orchestra.run.smithi129.stdout:48 files removed 2024-04-28T12:31:27.410 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-28T12:31:27.584 INFO:teuthology.orchestra.run.smithi106.stdout:48 files removed 2024-04-28T12:31:27.622 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-28T12:31:28.491 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 96 kB/s | 84 kB 00:00 2024-04-28T12:31:28.616 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 88 kB/s | 84 kB 00:00 2024-04-28T12:31:28.962 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 92 kB/s | 84 kB 00:00 2024-04-28T12:31:29.309 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 15 kB/s | 12 kB 00:00 2024-04-28T12:31:29.392 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-04-28T12:31:29.962 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 13 kB/s | 12 kB 00:00 2024-04-28T12:31:30.026 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-04-28T12:31:30.192 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 2.5 kB/s | 1.9 kB 00:00 2024-04-28T12:31:30.484 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-04-28T12:31:30.896 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-04-28T12:31:32.326 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.0 MB 00:02 2024-04-28T12:31:32.569 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-28T12:31:32.574 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-04-28T12:31:34.887 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-28T12:31:36.359 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 19 MB 00:02 2024-04-28T12:31:36.803 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.2 MB 00:00 2024-04-28T12:31:38.295 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 16 kB 00:00 2024-04-28T12:31:39.355 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-28T12:31:39.454 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-28T12:31:41.188 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-28T12:31:41.527 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.2 MB 00:01 2024-04-28T12:31:43.153 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-28T12:31:43.315 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-28T12:31:44.354 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-28T12:31:44.413 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-28T12:31:46.493 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:31:46.493 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:31:46.504 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-28T12:31:46.505 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-28T12:31:46.505 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-28T12:31:46.506 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-28T12:31:46.508 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-28T12:31:46.510 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-28T12:31:46.511 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-28T12:31:46.571 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-04-28T12:31:46.595 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-28T12:31:46.596 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-28T12:31:46.597 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-28T12:31:46.598 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-28T12:31:46.598 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-28T12:31:46.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-28T12:31:46.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-28T12:31:46.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-28T12:31:46.598 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-28T12:31:46.598 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-28T12:31:46.598 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:31:46.598 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-28T12:31:46.599 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-28T12:31:46.600 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-28T12:31:46.601 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-28T12:31:46.602 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-28T12:31:46.603 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-28T12:31:46.604 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-28T12:31:46.605 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-28T12:31:46.606 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout:Install 148 Packages 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 4 Packages 2024-04-28T12:31:46.607 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:31:46.608 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 222 M 2024-04-28T12:31:46.608 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-04-28T12:31:47.606 INFO:teuthology.orchestra.run.smithi171.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-04-28T12:31:48.848 INFO:teuthology.orchestra.run.smithi171.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 716 kB/s | 889 kB 00:01 2024-04-28T12:31:49.148 INFO:teuthology.orchestra.run.smithi171.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 484 kB/s | 145 kB 00:00 2024-04-28T12:31:49.337 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-28T12:31:49.423 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-04-28T12:31:51.405 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:31:51.406 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:31:51.417 INFO:teuthology.orchestra.run.smithi129.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-28T12:31:51.418 INFO:teuthology.orchestra.run.smithi129.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-28T12:31:51.418 INFO:teuthology.orchestra.run.smithi129.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-28T12:31:51.419 INFO:teuthology.orchestra.run.smithi129.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-28T12:31:51.420 INFO:teuthology.orchestra.run.smithi129.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-28T12:31:51.423 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-28T12:31:51.424 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-28T12:31:51.483 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-04-28T12:31:51.500 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-28T12:31:51.500 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-04-28T12:31:51.500 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-28T12:31:51.500 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-04-28T12:31:51.500 INFO:teuthology.orchestra.run.smithi129.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-28T12:31:51.500 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-28T12:31:51.501 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-28T12:31:51.502 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-28T12:31:51.503 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-28T12:31:51.504 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-28T12:31:51.505 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-28T12:31:51.506 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-28T12:31:51.507 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-28T12:31:51.508 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-28T12:31:51.509 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-28T12:31:51.510 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-28T12:31:51.511 INFO:teuthology.orchestra.run.smithi129.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout:======================================================================================= 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout:Install 148 Packages 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 4 Packages 2024-04-28T12:31:51.512 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:31:51.513 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 222 M 2024-04-28T12:31:51.513 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-04-28T12:31:51.749 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:31:51.750 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-28T12:31:51.765 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-28T12:31:51.766 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-28T12:31:51.767 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-28T12:31:51.769 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-28T12:31:51.771 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-28T12:31:51.774 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-28T12:31:51.775 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-28T12:31:51.844 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-28T12:31:51.862 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-28T12:31:51.862 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-04-28T12:31:51.862 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-28T12:31:51.862 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-04-28T12:31:51.862 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-28T12:31:51.862 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-28T12:31:51.862 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:31:51.862 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-28T12:31:51.862 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-28T12:31:51.863 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-28T12:31:51.864 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-28T12:31:51.865 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-28T12:31:51.866 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-28T12:31:51.866 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-28T12:31:51.866 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-28T12:31:51.866 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-28T12:31:51.866 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-28T12:31:51.867 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-28T12:31:51.867 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-28T12:31:51.867 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-28T12:31:51.867 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-28T12:31:51.867 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-28T12:31:51.867 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-28T12:31:51.868 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-28T12:31:51.868 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-28T12:31:51.868 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-28T12:31:51.868 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-28T12:31:51.868 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-28T12:31:51.868 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-28T12:31:51.868 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-28T12:31:51.868 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-28T12:31:51.868 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-28T12:31:51.869 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-28T12:31:51.870 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-28T12:31:51.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-28T12:31:51.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-28T12:31:51.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-28T12:31:51.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-28T12:31:51.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-28T12:31:51.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-28T12:31:51.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-28T12:31:51.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-28T12:31:51.871 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-28T12:31:51.872 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-28T12:31:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-28T12:31:51.874 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-28T12:31:51.875 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout:Install 148 Packages 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 4 Packages 2024-04-28T12:31:51.876 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:31:51.877 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 222 M 2024-04-28T12:31:51.877 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-04-28T12:31:52.680 INFO:teuthology.orchestra.run.smithi129.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-28T12:31:53.058 INFO:teuthology.orchestra.run.smithi106.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-28T12:31:53.073 INFO:teuthology.orchestra.run.smithi171.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 950 kB/s | 5.4 MB 00:05 2024-04-28T12:31:53.731 INFO:teuthology.orchestra.run.smithi129.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 847 kB/s | 889 kB 00:01 2024-04-28T12:31:53.790 INFO:teuthology.orchestra.run.smithi171.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 521 kB/s | 2.4 MB 00:04 2024-04-28T12:31:53.931 INFO:teuthology.orchestra.run.smithi129.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 727 kB/s | 145 kB 00:00 2024-04-28T12:31:54.575 INFO:teuthology.orchestra.run.smithi106.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 587 kB/s | 889 kB 00:01 2024-04-28T12:31:54.992 INFO:teuthology.orchestra.run.smithi106.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 349 kB/s | 145 kB 00:00 2024-04-28T12:31:55.506 INFO:teuthology.orchestra.run.smithi171.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 637 kB/s | 1.5 MB 00:02 2024-04-28T12:31:55.856 INFO:teuthology.orchestra.run.smithi129.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 1.2 MB/s | 2.4 MB 00:01 2024-04-28T12:31:57.056 INFO:teuthology.orchestra.run.smithi129.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 1.3 MB/s | 1.5 MB 00:01 2024-04-28T12:32:00.175 INFO:teuthology.orchestra.run.smithi106.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 728 kB/s | 5.4 MB 00:07 2024-04-28T12:32:01.983 INFO:teuthology.orchestra.run.smithi106.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 346 kB/s | 2.4 MB 00:06 2024-04-28T12:32:03.106 INFO:teuthology.orchestra.run.smithi129.stdout:(6/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 796 kB/s | 4.7 MB 00:06 2024-04-28T12:32:04.234 INFO:teuthology.orchestra.run.smithi106.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 382 kB/s | 1.5 MB 00:04 2024-04-28T12:32:06.206 INFO:teuthology.orchestra.run.smithi129.stdout:(7/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 394 kB/s | 5.4 MB 00:13 2024-04-28T12:32:06.531 INFO:teuthology.orchestra.run.smithi171.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 378 kB/s | 4.7 MB 00:12 2024-04-28T12:32:16.916 INFO:teuthology.orchestra.run.smithi106.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 322 kB/s | 4.7 MB 00:14 2024-04-28T12:32:28.488 INFO:teuthology.orchestra.run.smithi171.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 525 kB/s | 21 MB 00:41 2024-04-28T12:32:28.671 INFO:teuthology.orchestra.run.smithi171.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 138 kB/s | 25 kB 00:00 2024-04-28T12:32:37.616 INFO:teuthology.orchestra.run.smithi106.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 481 kB/s | 21 MB 00:44 2024-04-28T12:32:38.174 INFO:teuthology.orchestra.run.smithi106.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 45 kB/s | 25 kB 00:00 2024-04-28T12:32:38.870 INFO:teuthology.orchestra.run.smithi171.stdout:(10/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 397 kB/s | 17 MB 00:43 2024-04-28T12:32:39.370 INFO:teuthology.orchestra.run.smithi171.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 450 kB/s | 225 kB 00:00 2024-04-28T12:32:39.737 INFO:teuthology.orchestra.run.smithi171.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 74 kB/s | 27 kB 00:00 2024-04-28T12:32:40.096 INFO:teuthology.orchestra.run.smithi171.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 88 kB/s | 32 kB 00:00 2024-04-28T12:32:42.404 INFO:teuthology.orchestra.run.smithi171.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 324 kB/s | 747 kB 00:02 2024-04-28T12:32:42.895 INFO:teuthology.orchestra.run.smithi171.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 332 kB/s | 163 kB 00:00 2024-04-28T12:32:43.379 INFO:teuthology.orchestra.run.smithi171.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 263 kB/s | 127 kB 00:00 2024-04-28T12:32:44.904 INFO:teuthology.orchestra.run.smithi171.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 328 kB/s | 501 kB 00:01 2024-04-28T12:32:48.724 INFO:teuthology.orchestra.run.smithi106.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 341 kB/s | 11 MB 00:31 2024-04-28T12:32:49.648 INFO:teuthology.orchestra.run.smithi106.stdout:(11/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 379 kB/s | 17 MB 00:45 2024-04-28T12:32:49.769 INFO:teuthology.orchestra.run.smithi129.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 249 kB/s | 11 MB 00:43 2024-04-28T12:32:49.856 INFO:teuthology.orchestra.run.smithi106.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 130 kB/s | 27 kB 00:00 2024-04-28T12:32:49.977 INFO:teuthology.orchestra.run.smithi129.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 121 kB/s | 25 kB 00:00 2024-04-28T12:32:50.007 INFO:teuthology.orchestra.run.smithi106.stdout:(13/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 175 kB/s | 225 kB 00:01 2024-04-28T12:32:50.057 INFO:teuthology.orchestra.run.smithi106.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 159 kB/s | 32 kB 00:00 2024-04-28T12:32:50.924 INFO:teuthology.orchestra.run.smithi106.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 189 kB/s | 163 kB 00:00 2024-04-28T12:32:51.582 INFO:teuthology.orchestra.run.smithi106.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 193 kB/s | 127 kB 00:00 2024-04-28T12:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 360 kB/s | 21 MB 01:00 2024-04-28T12:32:52.657 INFO:teuthology.orchestra.run.smithi106.stdout:(17/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 282 kB/s | 747 kB 00:02 2024-04-28T12:32:53.074 INFO:teuthology.orchestra.run.smithi106.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 336 kB/s | 501 kB 00:01 2024-04-28T12:32:53.210 INFO:teuthology.orchestra.run.smithi129.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 250 kB/s | 225 kB 00:00 2024-04-28T12:32:53.275 INFO:teuthology.orchestra.run.smithi106.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 226 kB/s | 45 kB 00:00 2024-04-28T12:32:53.327 INFO:teuthology.orchestra.run.smithi129.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 232 kB/s | 27 kB 00:00 2024-04-28T12:32:53.544 INFO:teuthology.orchestra.run.smithi129.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 146 kB/s | 32 kB 00:00 2024-04-28T12:32:53.618 INFO:teuthology.orchestra.run.smithi106.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 396 kB/s | 135 kB 00:00 2024-04-28T12:32:54.033 INFO:teuthology.orchestra.run.smithi106.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 388 kB/s | 161 kB 00:00 2024-04-28T12:32:54.692 INFO:teuthology.orchestra.run.smithi106.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 488 kB/s | 321 kB 00:00 2024-04-28T12:32:55.351 INFO:teuthology.orchestra.run.smithi106.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 453 kB/s | 298 kB 00:00 2024-04-28T12:32:55.595 INFO:teuthology.orchestra.run.smithi171.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 514 kB/s | 5.4 MB 00:10 2024-04-28T12:32:55.701 INFO:teuthology.orchestra.run.smithi106.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 284 kB/s | 99 kB 00:00 2024-04-28T12:32:55.703 INFO:teuthology.orchestra.run.smithi171.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 417 kB/s | 45 kB 00:00 2024-04-28T12:32:55.894 INFO:teuthology.orchestra.run.smithi129.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 318 kB/s | 747 kB 00:02 2024-04-28T12:32:56.043 INFO:teuthology.orchestra.run.smithi106.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 250 kB/s | 85 kB 00:00 2024-04-28T12:32:56.128 INFO:teuthology.orchestra.run.smithi171.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 318 kB/s | 135 kB 00:00 2024-04-28T12:32:56.370 INFO:teuthology.orchestra.run.smithi171.stdout:(21/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 218 kB/s | 11 MB 00:49 2024-04-28T12:32:56.403 INFO:teuthology.orchestra.run.smithi129.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 321 kB/s | 163 kB 00:00 2024-04-28T12:32:56.629 INFO:teuthology.orchestra.run.smithi171.stdout:(22/152): python3-cephfs-19.0.0-3306.gbc612ffb. 323 kB/s | 161 kB 00:00 2024-04-28T12:32:57.212 INFO:teuthology.orchestra.run.smithi129.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 157 kB/s | 127 kB 00:00 2024-04-28T12:32:57.821 INFO:teuthology.orchestra.run.smithi171.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 250 kB/s | 298 kB 00:01 2024-04-28T12:32:58.154 INFO:teuthology.orchestra.run.smithi171.stdout:(24/152): python3-rados-19.0.0-3306.gbc612ffb.e 180 kB/s | 321 kB 00:01 2024-04-28T12:32:58.263 INFO:teuthology.orchestra.run.smithi171.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 225 kB/s | 99 kB 00:00 2024-04-28T12:32:58.805 INFO:teuthology.orchestra.run.smithi171.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 131 kB/s | 85 kB 00:00 2024-04-28T12:32:59.287 INFO:teuthology.orchestra.run.smithi129.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 241 kB/s | 501 kB 00:02 2024-04-28T12:32:59.930 INFO:teuthology.orchestra.run.smithi171.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 152 kB/s | 171 kB 00:01 2024-04-28T12:33:00.239 INFO:teuthology.orchestra.run.smithi171.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 111 kB/s | 34 kB 00:00 2024-04-28T12:33:01.214 INFO:teuthology.orchestra.run.smithi171.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 146 kB/s | 142 kB 00:00 2024-04-28T12:33:02.128 INFO:teuthology.orchestra.run.smithi129.stdout:(18/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 292 kB/s | 17 MB 00:59 2024-04-28T12:33:02.820 INFO:teuthology.orchestra.run.smithi129.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 65 kB/s | 45 kB 00:00 2024-04-28T12:33:03.312 INFO:teuthology.orchestra.run.smithi129.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 275 kB/s | 135 kB 00:00 2024-04-28T12:33:03.829 INFO:teuthology.orchestra.run.smithi129.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 313 kB/s | 161 kB 00:00 2024-04-28T12:33:04.929 INFO:teuthology.orchestra.run.smithi129.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 292 kB/s | 321 kB 00:01 2024-04-28T12:33:06.346 INFO:teuthology.orchestra.run.smithi129.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 210 kB/s | 298 kB 00:01 2024-04-28T12:33:06.863 INFO:teuthology.orchestra.run.smithi129.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 192 kB/s | 99 kB 00:00 2024-04-28T12:33:06.875 INFO:teuthology.orchestra.run.smithi106.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 292 kB/s | 3.1 MB 00:10 2024-04-28T12:33:07.271 INFO:teuthology.orchestra.run.smithi129.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 209 kB/s | 85 kB 00:00 2024-04-28T12:33:07.680 INFO:teuthology.orchestra.run.smithi171.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 336 kB/s | 3.1 MB 00:09 2024-04-28T12:33:07.750 INFO:teuthology.orchestra.run.smithi106.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 195 kB/s | 171 kB 00:00 2024-04-28T12:33:07.934 INFO:teuthology.orchestra.run.smithi106.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 187 kB/s | 34 kB 00:00 2024-04-28T12:33:08.509 INFO:teuthology.orchestra.run.smithi106.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 247 kB/s | 142 kB 00:00 2024-04-28T12:33:10.951 INFO:teuthology.orchestra.run.smithi106.stdout:(30/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 301 kB/s | 5.4 MB 00:18 2024-04-28T12:33:17.243 INFO:teuthology.orchestra.run.smithi106.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 415 kB/s | 3.5 MB 00:08 2024-04-28T12:33:17.893 INFO:teuthology.orchestra.run.smithi106.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 383 kB/s | 249 kB 00:00 2024-04-28T12:33:18.076 INFO:teuthology.orchestra.run.smithi106.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 270 kB/s | 49 kB 00:00 2024-04-28T12:33:18.187 INFO:teuthology.orchestra.run.smithi106.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 149 kB/s | 16 kB 00:00 2024-04-28T12:33:18.994 INFO:teuthology.orchestra.run.smithi106.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 339 kB/s | 273 kB 00:00 2024-04-28T12:33:20.704 INFO:teuthology.orchestra.run.smithi171.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 186 kB/s | 3.5 MB 00:19 2024-04-28T12:33:21.035 INFO:teuthology.orchestra.run.smithi106.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 374 kB/s | 762 kB 00:02 2024-04-28T12:33:21.144 INFO:teuthology.orchestra.run.smithi106.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 169 kB/s | 18 kB 00:00 2024-04-28T12:33:21.286 INFO:teuthology.orchestra.run.smithi106.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 572 kB/s | 80 kB 00:00 2024-04-28T12:33:21.469 INFO:teuthology.orchestra.run.smithi106.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-04-28T12:33:21.470 INFO:teuthology.orchestra.run.smithi129.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 223 kB/s | 3.1 MB 00:14 2024-04-28T12:33:21.528 INFO:teuthology.orchestra.run.smithi106.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-28T12:33:21.587 INFO:teuthology.orchestra.run.smithi106.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-28T12:33:21.778 INFO:teuthology.orchestra.run.smithi106.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 6.4 MB/s | 1.2 MB 00:00 2024-04-28T12:33:21.854 INFO:teuthology.orchestra.run.smithi106.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-28T12:33:21.913 INFO:teuthology.orchestra.run.smithi106.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-28T12:33:21.971 INFO:teuthology.orchestra.run.smithi106.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-28T12:33:22.030 INFO:teuthology.orchestra.run.smithi106.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-28T12:33:22.353 INFO:teuthology.orchestra.run.smithi129.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 193 kB/s | 171 kB 00:00 2024-04-28T12:33:22.504 INFO:teuthology.orchestra.run.smithi171.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 138 kB/s | 249 kB 00:01 2024-04-28T12:33:22.770 INFO:teuthology.orchestra.run.smithi129.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 82 kB/s | 34 kB 00:00 2024-04-28T12:33:22.779 INFO:teuthology.orchestra.run.smithi171.stdout:(33/152): ceph-mgr-diskprediction-local-19.0.0- 502 kB/s | 7.4 MB 00:15 2024-04-28T12:33:22.871 INFO:teuthology.orchestra.run.smithi171.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 180 kB/s | 16 kB 00:00 2024-04-28T12:33:22.872 INFO:teuthology.orchestra.run.smithi106.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 819 kB/s | 689 kB 00:00 2024-04-28T12:33:22.929 INFO:teuthology.orchestra.run.smithi171.stdout:(35/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 116 kB/s | 49 kB 00:00 2024-04-28T12:33:23.097 INFO:teuthology.orchestra.run.smithi106.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-04-28T12:33:23.205 INFO:teuthology.orchestra.run.smithi106.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 994 kB/s | 107 kB 00:00 2024-04-28T12:33:23.314 INFO:teuthology.orchestra.run.smithi106.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-04-28T12:33:23.363 INFO:teuthology.orchestra.run.smithi171.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 555 kB/s | 273 kB 00:00 2024-04-28T12:33:23.439 INFO:teuthology.orchestra.run.smithi106.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 2.5 MB/s | 313 kB 00:00 2024-04-28T12:33:23.472 INFO:teuthology.orchestra.run.smithi171.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 168 kB/s | 18 kB 00:00 2024-04-28T12:33:23.548 INFO:teuthology.orchestra.run.smithi106.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 303 kB/s | 33 kB 00:00 2024-04-28T12:33:23.562 INFO:teuthology.orchestra.run.smithi129.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 180 kB/s | 142 kB 00:00 2024-04-28T12:33:23.580 INFO:teuthology.orchestra.run.smithi171.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-04-28T12:33:23.689 INFO:teuthology.orchestra.run.smithi171.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-04-28T12:33:23.731 INFO:teuthology.orchestra.run.smithi171.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-28T12:33:23.781 INFO:teuthology.orchestra.run.smithi171.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-28T12:33:23.865 INFO:teuthology.orchestra.run.smithi171.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-28T12:33:23.915 INFO:teuthology.orchestra.run.smithi171.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-28T12:33:23.957 INFO:teuthology.orchestra.run.smithi106.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.3 MB/s | 3.0 MB 00:00 2024-04-28T12:33:23.965 INFO:teuthology.orchestra.run.smithi171.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-28T12:33:24.024 INFO:teuthology.orchestra.run.smithi171.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-28T12:33:24.074 INFO:teuthology.orchestra.run.smithi106.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 157 kB/s | 18 kB 00:00 2024-04-28T12:33:24.074 INFO:teuthology.orchestra.run.smithi171.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-28T12:33:24.174 INFO:teuthology.orchestra.run.smithi171.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-04-28T12:33:24.182 INFO:teuthology.orchestra.run.smithi106.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 11 kB 00:00 2024-04-28T12:33:24.292 INFO:teuthology.orchestra.run.smithi171.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 943 kB 00:00 2024-04-28T12:33:24.358 INFO:teuthology.orchestra.run.smithi171.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-28T12:33:24.409 INFO:teuthology.orchestra.run.smithi171.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-28T12:33:24.449 INFO:teuthology.orchestra.run.smithi106.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 602 kB/s | 160 kB 00:00 2024-04-28T12:33:24.492 INFO:teuthology.orchestra.run.smithi171.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-28T12:33:24.543 INFO:teuthology.orchestra.run.smithi171.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-04-28T12:33:24.558 INFO:teuthology.orchestra.run.smithi106.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 420 kB/s | 45 kB 00:00 2024-04-28T12:33:24.701 INFO:teuthology.orchestra.run.smithi171.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-28T12:33:24.752 INFO:teuthology.orchestra.run.smithi171.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-28T12:33:24.794 INFO:teuthology.orchestra.run.smithi171.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-04-28T12:33:24.861 INFO:teuthology.orchestra.run.smithi171.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-28T12:33:24.919 INFO:teuthology.orchestra.run.smithi171.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-04-28T12:33:25.011 INFO:teuthology.orchestra.run.smithi171.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-28T12:33:25.095 INFO:teuthology.orchestra.run.smithi171.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-28T12:33:25.145 INFO:teuthology.orchestra.run.smithi171.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-04-28T12:33:25.204 INFO:teuthology.orchestra.run.smithi171.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-04-28T12:33:25.279 INFO:teuthology.orchestra.run.smithi171.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-28T12:33:25.346 INFO:teuthology.orchestra.run.smithi171.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-28T12:33:25.388 INFO:teuthology.orchestra.run.smithi171.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 847 kB/s | 35 kB 00:00 2024-04-28T12:33:25.471 INFO:teuthology.orchestra.run.smithi129.stdout:(30/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 210 kB/s | 5.4 MB 00:26 2024-04-28T12:33:25.533 INFO:teuthology.orchestra.run.smithi106.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 680 kB/s | 662 kB 00:00 2024-04-28T12:33:25.605 INFO:teuthology.orchestra.run.smithi171.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-04-28T12:33:25.659 INFO:teuthology.orchestra.run.smithi106.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 582 kB 00:00 2024-04-28T12:33:25.775 INFO:teuthology.orchestra.run.smithi106.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 219 kB/s | 25 kB 00:00 2024-04-28T12:33:25.814 INFO:teuthology.orchestra.run.smithi171.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-28T12:33:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 382 kB/s | 47 kB 00:00 2024-04-28T12:33:25.939 INFO:teuthology.orchestra.run.smithi171.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-28T12:33:26.048 INFO:teuthology.orchestra.run.smithi171.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-28T12:33:26.115 INFO:teuthology.orchestra.run.smithi171.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-28T12:33:26.234 INFO:teuthology.orchestra.run.smithi106.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 879 kB/s | 292 kB 00:00 2024-04-28T12:33:26.348 INFO:teuthology.orchestra.run.smithi171.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-28T12:33:26.368 INFO:teuthology.orchestra.run.smithi106.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-28T12:33:26.457 INFO:teuthology.orchestra.run.smithi171.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-28T12:33:26.485 INFO:teuthology.orchestra.run.smithi106.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 303 kB/s | 35 kB 00:00 2024-04-28T12:33:26.524 INFO:teuthology.orchestra.run.smithi171.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-28T12:33:26.783 INFO:teuthology.orchestra.run.smithi171.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-04-28T12:33:26.866 INFO:teuthology.orchestra.run.smithi171.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-28T12:33:26.942 INFO:teuthology.orchestra.run.smithi171.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-28T12:33:26.992 INFO:teuthology.orchestra.run.smithi171.stdout:(76/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 188 kB/s | 762 kB 00:04 2024-04-28T12:33:27.042 INFO:teuthology.orchestra.run.smithi171.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-04-28T12:33:27.084 INFO:teuthology.orchestra.run.smithi171.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 208 kB/s | 19 kB 00:00 2024-04-28T12:33:27.126 INFO:teuthology.orchestra.run.smithi171.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-28T12:33:27.152 INFO:teuthology.orchestra.run.smithi171.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 517 kB/s | 35 kB 00:00 2024-04-28T12:33:27.227 INFO:teuthology.orchestra.run.smithi171.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-28T12:33:27.319 INFO:teuthology.orchestra.run.smithi171.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-28T12:33:27.428 INFO:teuthology.orchestra.run.smithi171.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-04-28T12:33:27.453 INFO:teuthology.orchestra.run.smithi171.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 577 kB/s | 77 kB 00:00 2024-04-28T12:33:27.495 INFO:teuthology.orchestra.run.smithi171.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 633 kB/s | 42 kB 00:00 2024-04-28T12:33:27.545 INFO:teuthology.orchestra.run.smithi171.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-28T12:33:27.587 INFO:teuthology.orchestra.run.smithi171.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-28T12:33:27.637 INFO:teuthology.orchestra.run.smithi171.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-28T12:33:27.654 INFO:teuthology.orchestra.run.smithi171.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-04-28T12:33:27.696 INFO:teuthology.orchestra.run.smithi171.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-04-28T12:33:27.746 INFO:teuthology.orchestra.run.smithi171.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-04-28T12:33:27.797 INFO:teuthology.orchestra.run.smithi171.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-04-28T12:33:27.872 INFO:teuthology.orchestra.run.smithi171.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-28T12:33:27.914 INFO:teuthology.orchestra.run.smithi171.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 859 kB/s | 36 kB 00:00 2024-04-28T12:33:27.973 INFO:teuthology.orchestra.run.smithi171.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-28T12:33:28.065 INFO:teuthology.orchestra.run.smithi171.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.3 MB/s | 305 kB 00:00 2024-04-28T12:33:28.240 INFO:teuthology.orchestra.run.smithi171.stdout:(97/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-28T12:33:28.290 INFO:teuthology.orchestra.run.smithi171.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 256 kB/s | 58 kB 00:00 2024-04-28T12:33:28.324 INFO:teuthology.orchestra.run.smithi171.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-28T12:33:28.383 INFO:teuthology.orchestra.run.smithi171.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-04-28T12:33:28.441 INFO:teuthology.orchestra.run.smithi171.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 422 kB/s | 64 kB 00:00 2024-04-28T12:33:28.466 INFO:teuthology.orchestra.run.smithi171.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-04-28T12:33:28.567 INFO:teuthology.orchestra.run.smithi171.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-04-28T12:33:28.958 INFO:teuthology.orchestra.run.smithi171.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-28T12:33:29.009 INFO:teuthology.orchestra.run.smithi171.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 744 kB/s | 309 kB 00:00 2024-04-28T12:33:29.026 INFO:teuthology.orchestra.run.smithi171.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 290 kB/s | 19 kB 00:00 2024-04-28T12:33:29.268 INFO:teuthology.orchestra.run.smithi171.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 107 kB/s | 26 kB 00:00 2024-04-28T12:33:29.352 INFO:teuthology.orchestra.run.smithi171.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-28T12:33:29.368 INFO:teuthology.orchestra.run.smithi171.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 493 kB/s | 49 kB 00:00 2024-04-28T12:33:29.427 INFO:teuthology.orchestra.run.smithi171.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 899 kB/s | 67 kB 00:00 2024-04-28T12:33:29.453 INFO:teuthology.orchestra.run.smithi171.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-28T12:33:29.519 INFO:teuthology.orchestra.run.smithi171.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-04-28T12:33:29.553 INFO:teuthology.orchestra.run.smithi171.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-28T12:33:29.603 INFO:teuthology.orchestra.run.smithi171.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 527 kB/s | 43 kB 00:00 2024-04-28T12:33:29.628 INFO:teuthology.orchestra.run.smithi171.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 427 kB/s | 32 kB 00:00 2024-04-28T12:33:29.687 INFO:teuthology.orchestra.run.smithi171.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-04-28T12:33:29.712 INFO:teuthology.orchestra.run.smithi171.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-28T12:33:29.771 INFO:teuthology.orchestra.run.smithi171.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-28T12:33:29.796 INFO:teuthology.orchestra.run.smithi171.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-28T12:33:29.846 INFO:teuthology.orchestra.run.smithi171.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 472 kB/s | 35 kB 00:00 2024-04-28T12:33:29.880 INFO:teuthology.orchestra.run.smithi171.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-04-28T12:33:29.972 INFO:teuthology.orchestra.run.smithi171.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-28T12:33:29.997 INFO:teuthology.orchestra.run.smithi171.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-28T12:33:30.048 INFO:teuthology.orchestra.run.smithi171.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-28T12:33:30.073 INFO:teuthology.orchestra.run.smithi171.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-28T12:33:30.123 INFO:teuthology.orchestra.run.smithi171.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-04-28T12:33:30.148 INFO:teuthology.orchestra.run.smithi171.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-04-28T12:33:30.199 INFO:teuthology.orchestra.run.smithi171.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-28T12:33:30.257 INFO:teuthology.orchestra.run.smithi171.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-28T12:33:30.358 INFO:teuthology.orchestra.run.smithi171.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 292 kB/s | 46 kB 00:00 2024-04-28T12:33:30.391 INFO:teuthology.orchestra.run.smithi171.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 591 kB/s | 79 kB 00:00 2024-04-28T12:33:30.425 INFO:teuthology.orchestra.run.smithi171.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 865 kB/s | 58 kB 00:00 2024-04-28T12:33:30.467 INFO:teuthology.orchestra.run.smithi171.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-28T12:33:30.492 INFO:teuthology.orchestra.run.smithi171.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-04-28T12:33:30.542 INFO:teuthology.orchestra.run.smithi171.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-28T12:33:30.576 INFO:teuthology.orchestra.run.smithi171.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-04-28T12:33:30.627 INFO:teuthology.orchestra.run.smithi171.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 366 kB/s | 31 kB 00:00 2024-04-28T12:33:30.660 INFO:teuthology.orchestra.run.smithi171.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-28T12:33:30.702 INFO:teuthology.orchestra.run.smithi106.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.1 MB/s | 4.7 MB 00:04 2024-04-28T12:33:30.710 INFO:teuthology.orchestra.run.smithi171.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 708 kB/s | 59 kB 00:00 2024-04-28T12:33:30.760 INFO:teuthology.orchestra.run.smithi171.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 357 kB/s | 36 kB 00:00 2024-04-28T12:33:30.786 INFO:teuthology.orchestra.run.smithi171.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-04-28T12:33:30.836 INFO:teuthology.orchestra.run.smithi171.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-28T12:33:30.861 INFO:teuthology.orchestra.run.smithi171.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-28T12:33:30.912 INFO:teuthology.orchestra.run.smithi171.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-28T12:33:30.937 INFO:teuthology.orchestra.run.smithi171.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-04-28T12:33:30.979 INFO:teuthology.orchestra.run.smithi171.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-28T12:33:31.046 INFO:teuthology.orchestra.run.smithi171.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-28T12:33:34.602 INFO:teuthology.orchestra.run.smithi106.stdout:(66/152): ceph-mgr-diskprediction-local-19.0.0- 321 kB/s | 7.4 MB 00:23 2024-04-28T12:33:35.627 INFO:teuthology.orchestra.run.smithi106.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:01 2024-04-28T12:33:35.760 INFO:teuthology.orchestra.run.smithi106.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-28T12:33:35.869 INFO:teuthology.orchestra.run.smithi106.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-28T12:33:37.269 INFO:teuthology.orchestra.run.smithi106.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 690 kB/s | 4.4 MB 00:06 2024-04-28T12:33:38.802 INFO:teuthology.orchestra.run.smithi106.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 674 kB/s | 1.0 MB 00:01 2024-04-28T12:33:39.044 INFO:teuthology.orchestra.run.smithi106.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 363 kB/s | 87 kB 00:00 2024-04-28T12:33:39.295 INFO:teuthology.orchestra.run.smithi106.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 1.7 MB/s | 5.8 MB 00:03 2024-04-28T12:33:39.612 INFO:teuthology.orchestra.run.smithi171.stdout:(148/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 375 kB/s | 3.1 MB 00:08 2024-04-28T12:33:39.788 INFO:teuthology.orchestra.run.smithi171.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 1.3 MB 00:00 2024-04-28T12:33:39.880 INFO:teuthology.orchestra.run.smithi171.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-28T12:33:40.053 INFO:teuthology.orchestra.run.smithi106.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 322 kB/s | 244 kB 00:00 2024-04-28T12:33:40.287 INFO:teuthology.orchestra.run.smithi106.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.0 MB/s | 248 kB 00:00 2024-04-28T12:33:40.412 INFO:teuthology.orchestra.run.smithi106.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 206 kB/s | 26 kB 00:00 2024-04-28T12:33:40.529 INFO:teuthology.orchestra.run.smithi106.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 165 kB/s | 19 kB 00:00 2024-04-28T12:33:40.656 INFO:teuthology.orchestra.run.smithi106.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-28T12:33:40.763 INFO:teuthology.orchestra.run.smithi106.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 328 kB/s | 35 kB 00:00 2024-04-28T12:33:41.002 INFO:teuthology.orchestra.run.smithi129.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 208 kB/s | 3.5 MB 00:17 2024-04-28T12:33:41.421 INFO:teuthology.orchestra.run.smithi106.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.8 MB/s | 5.1 MB 00:00 2024-04-28T12:33:41.788 INFO:teuthology.orchestra.run.smithi106.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 685 kB/s | 250 kB 00:00 2024-04-28T12:33:42.180 INFO:teuthology.orchestra.run.smithi106.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 568 kB/s | 222 kB 00:00 2024-04-28T12:33:42.413 INFO:teuthology.orchestra.run.smithi106.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 332 kB/s | 77 kB 00:00 2024-04-28T12:33:42.461 INFO:teuthology.orchestra.run.smithi129.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 171 kB/s | 249 kB 00:01 2024-04-28T12:33:42.564 INFO:teuthology.orchestra.run.smithi106.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 282 kB/s | 42 kB 00:00 2024-04-28T12:33:42.742 INFO:teuthology.orchestra.run.smithi106.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-04-28T12:33:42.870 INFO:teuthology.orchestra.run.smithi129.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 121 kB/s | 49 kB 00:00 2024-04-28T12:33:42.891 INFO:teuthology.orchestra.run.smithi106.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-28T12:33:43.012 INFO:teuthology.orchestra.run.smithi129.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 116 kB/s | 16 kB 00:00 2024-04-28T12:33:43.042 INFO:teuthology.orchestra.run.smithi106.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-04-28T12:33:43.167 INFO:teuthology.orchestra.run.smithi106.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 409 kB/s | 51 kB 00:00 2024-04-28T12:33:43.301 INFO:teuthology.orchestra.run.smithi106.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 404 kB/s | 54 kB 00:00 2024-04-28T12:33:45.095 INFO:teuthology.orchestra.run.smithi129.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 131 kB/s | 273 kB 00:02 2024-04-28T12:33:47.904 INFO:teuthology.orchestra.run.smithi171.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 205 kB/s | 3.4 MB 00:16 2024-04-28T12:33:50.370 INFO:teuthology.orchestra.run.smithi129.stdout:(36/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 145 kB/s | 762 kB 00:05 2024-04-28T12:33:50.620 INFO:teuthology.orchestra.run.smithi129.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 73 kB/s | 18 kB 00:00 2024-04-28T12:33:51.159 INFO:teuthology.orchestra.run.smithi106.stdout:(90/152): python3-babel-2.9.1-2.el9.noarch.rpm 504 kB/s | 6.0 MB 00:12 2024-04-28T12:33:51.162 INFO:teuthology.orchestra.run.smithi129.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 148 kB/s | 80 kB 00:00 2024-04-28T12:33:51.301 INFO:teuthology.orchestra.run.smithi106.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 296 kB/s | 42 kB 00:00 2024-04-28T12:33:51.443 INFO:teuthology.orchestra.run.smithi106.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 313 kB/s | 44 kB 00:00 2024-04-28T12:33:51.576 INFO:teuthology.orchestra.run.smithi106.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-04-28T12:33:51.693 INFO:teuthology.orchestra.run.smithi106.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 309 kB/s | 36 kB 00:00 2024-04-28T12:33:51.802 INFO:teuthology.orchestra.run.smithi106.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 440 kB/s | 47 kB 00:00 2024-04-28T12:33:51.918 INFO:teuthology.orchestra.run.smithi106.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-28T12:33:52.019 INFO:teuthology.orchestra.run.smithi106.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 581 kB/s | 58 kB 00:00 2024-04-28T12:33:52.121 INFO:teuthology.orchestra.run.smithi129.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 838 kB/s | 803 kB 00:00 2024-04-28T12:33:52.127 INFO:teuthology.orchestra.run.smithi106.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 870 kB/s | 94 kB 00:00 2024-04-28T12:33:52.244 INFO:teuthology.orchestra.run.smithi106.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 549 kB/s | 64 kB 00:00 2024-04-28T12:33:52.321 INFO:teuthology.orchestra.run.smithi129.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 977 kB/s | 195 kB 00:00 2024-04-28T12:33:52.362 INFO:teuthology.orchestra.run.smithi106.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 386 kB/s | 45 kB 00:00 2024-04-28T12:33:52.478 INFO:teuthology.orchestra.run.smithi106.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 192 kB/s | 22 kB 00:00 2024-04-28T12:33:52.629 INFO:teuthology.orchestra.run.smithi106.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-04-28T12:33:52.646 INFO:teuthology.orchestra.run.smithi129.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 779 kB/s | 253 kB 00:00 2024-04-28T12:33:53.129 INFO:teuthology.orchestra.run.smithi106.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-28T12:33:53.280 INFO:teuthology.orchestra.run.smithi106.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.0 MB/s | 309 kB 00:00 2024-04-28T12:33:53.338 INFO:teuthology.orchestra.run.smithi106.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 339 kB/s | 19 kB 00:00 2024-04-28T12:33:54.038 INFO:teuthology.orchestra.run.smithi129.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 902 kB/s | 1.2 MB 00:01 2024-04-28T12:33:54.180 INFO:teuthology.orchestra.run.smithi129.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 752 kB/s | 106 kB 00:00 2024-04-28T12:33:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 141 kB/s | 135 kB 00:00 2024-04-28T12:33:55.381 INFO:teuthology.orchestra.run.smithi129.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 517 kB/s | 125 kB 00:00 2024-04-28T12:33:55.706 INFO:teuthology.orchestra.run.smithi129.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 661 kB/s | 215 kB 00:00 2024-04-28T12:33:56.255 INFO:teuthology.orchestra.run.smithi106.stdout:(106/152): libarrow-9.0.0-11.el9.x86_64.rpm 1.5 MB/s | 4.4 MB 00:02 2024-04-28T12:33:56.256 INFO:teuthology.orchestra.run.smithi129.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 689 kB 00:00 2024-04-28T12:33:56.347 INFO:teuthology.orchestra.run.smithi106.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 286 kB/s | 26 kB 00:00 2024-04-28T12:33:56.589 INFO:teuthology.orchestra.run.smithi106.stdout:(108/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.2 MB/s | 16 MB 00:13 2024-04-28T12:33:56.631 INFO:teuthology.orchestra.run.smithi129.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-04-28T12:33:56.639 INFO:teuthology.orchestra.run.smithi106.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 167 kB/s | 49 kB 00:00 2024-04-28T12:33:56.698 INFO:teuthology.orchestra.run.smithi129.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-28T12:33:56.765 INFO:teuthology.orchestra.run.smithi129.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-04-28T12:33:56.899 INFO:teuthology.orchestra.run.smithi129.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-28T12:33:56.948 INFO:teuthology.orchestra.run.smithi106.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 637 kB/s | 151 kB 00:00 2024-04-28T12:33:56.974 INFO:teuthology.orchestra.run.smithi129.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 436 kB/s | 33 kB 00:00 2024-04-28T12:33:56.990 INFO:teuthology.orchestra.run.smithi106.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 169 kB/s | 67 kB 00:00 2024-04-28T12:33:57.332 INFO:teuthology.orchestra.run.smithi106.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-04-28T12:33:57.424 INFO:teuthology.orchestra.run.smithi106.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 478 kB/s | 43 kB 00:00 2024-04-28T12:33:57.524 INFO:teuthology.orchestra.run.smithi106.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 325 kB/s | 32 kB 00:00 2024-04-28T12:33:57.625 INFO:teuthology.orchestra.run.smithi106.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2024-04-28T12:33:57.758 INFO:teuthology.orchestra.run.smithi106.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-28T12:33:57.791 INFO:teuthology.orchestra.run.smithi129.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.7 MB/s | 3.0 MB 00:00 2024-04-28T12:33:57.866 INFO:teuthology.orchestra.run.smithi129.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 243 kB/s | 18 kB 00:00 2024-04-28T12:33:57.958 INFO:teuthology.orchestra.run.smithi129.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-04-28T12:33:58.000 INFO:teuthology.orchestra.run.smithi106.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-04-28T12:33:58.042 INFO:teuthology.orchestra.run.smithi129.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-28T12:33:58.117 INFO:teuthology.orchestra.run.smithi129.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 604 kB/s | 45 kB 00:00 2024-04-28T12:33:58.142 INFO:teuthology.orchestra.run.smithi106.stdout:(118/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 703 kB/s | 839 kB 00:01 2024-04-28T12:33:58.217 INFO:teuthology.orchestra.run.smithi106.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 476 kB/s | 35 kB 00:00 2024-04-28T12:33:58.242 INFO:teuthology.orchestra.run.smithi129.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-28T12:33:58.376 INFO:teuthology.orchestra.run.smithi129.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-04-28T12:33:58.443 INFO:teuthology.orchestra.run.smithi129.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-04-28T12:33:58.459 INFO:teuthology.orchestra.run.smithi106.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 854 kB/s | 206 kB 00:00 2024-04-28T12:33:58.519 INFO:teuthology.orchestra.run.smithi129.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 634 kB/s | 47 kB 00:00 2024-04-28T12:33:58.602 INFO:teuthology.orchestra.run.smithi129.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-28T12:33:58.668 INFO:teuthology.orchestra.run.smithi106.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 455 kB/s | 303 kB 00:00 2024-04-28T12:33:58.685 INFO:teuthology.orchestra.run.smithi129.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-28T12:33:58.761 INFO:teuthology.orchestra.run.smithi129.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 470 kB/s | 35 kB 00:00 2024-04-28T12:33:59.018 INFO:teuthology.orchestra.run.smithi106.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 414 kB/s | 144 kB 00:00 2024-04-28T12:33:59.093 INFO:teuthology.orchestra.run.smithi106.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 144 kB/s | 11 kB 00:00 2024-04-28T12:33:59.177 INFO:teuthology.orchestra.run.smithi106.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 214 kB/s | 18 kB 00:00 2024-04-28T12:33:59.252 INFO:teuthology.orchestra.run.smithi106.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 312 kB/s | 23 kB 00:00 2024-04-28T12:33:59.319 INFO:teuthology.orchestra.run.smithi106.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 295 kB/s | 19 kB 00:00 2024-04-28T12:33:59.386 INFO:teuthology.orchestra.run.smithi106.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 298 kB/s | 20 kB 00:00 2024-04-28T12:33:59.672 INFO:teuthology.orchestra.run.smithi129.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.2 MB/s | 4.7 MB 00:00 2024-04-28T12:34:00.772 INFO:teuthology.orchestra.run.smithi129.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-04-28T12:34:01.372 INFO:teuthology.orchestra.run.smithi129.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-04-28T12:34:01.689 INFO:teuthology.orchestra.run.smithi129.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 2.8 MB/s | 911 kB 00:00 2024-04-28T12:34:01.764 INFO:teuthology.orchestra.run.smithi129.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-28T12:34:01.903 INFO:teuthology.orchestra.run.smithi106.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 606 kB/s | 2.0 MB 00:03 2024-04-28T12:34:02.036 INFO:teuthology.orchestra.run.smithi106.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 395 kB/s | 1.0 MB 00:02 2024-04-28T12:34:02.062 INFO:teuthology.orchestra.run.smithi106.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 294 kB/s | 46 kB 00:00 2024-04-28T12:34:02.195 INFO:teuthology.orchestra.run.smithi106.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 436 kB/s | 58 kB 00:00 2024-04-28T12:34:02.229 INFO:teuthology.orchestra.run.smithi106.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 412 kB/s | 79 kB 00:00 2024-04-28T12:34:02.296 INFO:teuthology.orchestra.run.smithi106.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 249 kB/s | 16 kB 00:00 2024-04-28T12:34:02.505 INFO:teuthology.orchestra.run.smithi106.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 433 kB/s | 90 kB 00:00 2024-04-28T12:34:02.621 INFO:teuthology.orchestra.run.smithi106.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 243 kB/s | 28 kB 00:00 2024-04-28T12:34:02.722 INFO:teuthology.orchestra.run.smithi106.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 310 kB/s | 31 kB 00:00 2024-04-28T12:34:02.755 INFO:teuthology.orchestra.run.smithi106.stdout:(137/152): python3-pecan-1.4.2-3.el9.noarch.rpm 487 kB/s | 272 kB 00:00 2024-04-28T12:34:02.848 INFO:teuthology.orchestra.run.smithi106.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 650 kB/s | 59 kB 00:00 2024-04-28T12:34:02.939 INFO:teuthology.orchestra.run.smithi106.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 394 kB/s | 36 kB 00:00 2024-04-28T12:34:02.973 INFO:teuthology.orchestra.run.smithi106.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 753 kB/s | 188 kB 00:00 2024-04-28T12:34:03.082 INFO:teuthology.orchestra.run.smithi106.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 361 kB/s | 51 kB 00:00 2024-04-28T12:34:03.240 INFO:teuthology.orchestra.run.smithi106.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 570 kB/s | 90 kB 00:00 2024-04-28T12:34:03.349 INFO:teuthology.orchestra.run.smithi106.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 613 kB/s | 230 kB 00:00 2024-04-28T12:34:03.416 INFO:teuthology.orchestra.run.smithi106.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 301 kB/s | 20 kB 00:00 2024-04-28T12:34:03.439 INFO:teuthology.orchestra.run.smithi129.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 3.5 MB/s | 5.8 MB 00:01 2024-04-28T12:34:03.866 INFO:teuthology.orchestra.run.smithi106.stdout:(145/152): re2-20211101-3.el9.x86_64.rpm 427 kB/s | 192 kB 00:00 2024-04-28T12:34:03.873 INFO:teuthology.orchestra.run.smithi129.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.3 MB/s | 1.0 MB 00:00 2024-04-28T12:34:03.923 INFO:teuthology.orchestra.run.smithi129.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-28T12:34:03.983 INFO:teuthology.orchestra.run.smithi106.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 576 kB/s | 427 kB 00:00 2024-04-28T12:34:04.207 INFO:teuthology.orchestra.run.smithi129.stdout:(73/152): ceph-mgr-diskprediction-local-19.0.0- 196 kB/s | 7.4 MB 00:38 2024-04-28T12:34:04.624 INFO:teuthology.orchestra.run.smithi129.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 586 kB/s | 244 kB 00:00 2024-04-28T12:34:04.916 INFO:teuthology.orchestra.run.smithi129.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 850 kB/s | 248 kB 00:00 2024-04-28T12:34:04.999 INFO:teuthology.orchestra.run.smithi129.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 308 kB/s | 26 kB 00:00 2024-04-28T12:34:05.083 INFO:teuthology.orchestra.run.smithi129.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 230 kB/s | 19 kB 00:00 2024-04-28T12:34:05.200 INFO:teuthology.orchestra.run.smithi129.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-04-28T12:34:05.283 INFO:teuthology.orchestra.run.smithi129.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 417 kB/s | 35 kB 00:00 2024-04-28T12:34:06.725 INFO:teuthology.orchestra.run.smithi129.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.5 MB/s | 5.1 MB 00:01 2024-04-28T12:34:06.783 INFO:teuthology.orchestra.run.smithi106.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 555 kB/s | 1.6 MB 00:02 2024-04-28T12:34:06.850 INFO:teuthology.orchestra.run.smithi129.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-04-28T12:34:06.942 INFO:teuthology.orchestra.run.smithi129.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-28T12:34:07.018 INFO:teuthology.orchestra.run.smithi129.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-28T12:34:07.110 INFO:teuthology.orchestra.run.smithi129.stdout:(84/152): python3-babel-2.9.1-2.el9.noarch.rpm 1.9 MB/s | 6.0 MB 00:03 2024-04-28T12:34:07.135 INFO:teuthology.orchestra.run.smithi129.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-04-28T12:34:07.235 INFO:teuthology.orchestra.run.smithi129.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-28T12:34:07.361 INFO:teuthology.orchestra.run.smithi129.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-04-28T12:34:07.486 INFO:teuthology.orchestra.run.smithi129.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 717 kB/s | 269 kB 00:00 2024-04-28T12:34:07.553 INFO:teuthology.orchestra.run.smithi129.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 805 kB/s | 54 kB 00:00 2024-04-28T12:34:07.620 INFO:teuthology.orchestra.run.smithi129.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 197 kB/s | 51 kB 00:00 2024-04-28T12:34:07.686 INFO:teuthology.orchestra.run.smithi129.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 625 kB/s | 42 kB 00:00 2024-04-28T12:34:07.737 INFO:teuthology.orchestra.run.smithi129.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-04-28T12:34:07.812 INFO:teuthology.orchestra.run.smithi129.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-28T12:34:07.854 INFO:teuthology.orchestra.run.smithi129.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 863 kB/s | 36 kB 00:00 2024-04-28T12:34:07.906 INFO:teuthology.orchestra.run.smithi129.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-04-28T12:34:07.980 INFO:teuthology.orchestra.run.smithi129.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-28T12:34:08.030 INFO:teuthology.orchestra.run.smithi129.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-28T12:34:08.088 INFO:teuthology.orchestra.run.smithi129.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-28T12:34:08.139 INFO:teuthology.orchestra.run.smithi129.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-28T12:34:08.198 INFO:teuthology.orchestra.run.smithi129.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-04-28T12:34:08.923 INFO:teuthology.orchestra.run.smithi129.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-04-28T12:34:10.116 INFO:teuthology.orchestra.run.smithi106.stdout:(148/152): librados2-19.0.0-3306.gbc612ffb.el9. 567 kB/s | 3.4 MB 00:06 2024-04-28T12:34:10.508 INFO:teuthology.orchestra.run.smithi106.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 1.3 MB 00:00 2024-04-28T12:34:11.000 INFO:teuthology.orchestra.run.smithi106.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 2.4 MB 00:00 2024-04-28T12:34:11.073 INFO:teuthology.orchestra.run.smithi129.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 402 kB/s | 863 kB 00:02 2024-04-28T12:34:11.515 INFO:teuthology.orchestra.run.smithi129.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-28T12:34:11.640 INFO:teuthology.orchestra.run.smithi129.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-04-28T12:34:11.707 INFO:teuthology.orchestra.run.smithi129.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-04-28T12:34:12.082 INFO:teuthology.orchestra.run.smithi129.stdout:(106/152): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-28T12:34:12.191 INFO:teuthology.orchestra.run.smithi129.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-28T12:34:12.316 INFO:teuthology.orchestra.run.smithi129.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-04-28T12:34:12.442 INFO:teuthology.orchestra.run.smithi129.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-04-28T12:34:12.525 INFO:teuthology.orchestra.run.smithi129.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-28T12:34:12.633 INFO:teuthology.orchestra.run.smithi129.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-04-28T12:34:12.734 INFO:teuthology.orchestra.run.smithi129.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-28T12:34:12.793 INFO:teuthology.orchestra.run.smithi129.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-04-28T12:34:12.851 INFO:teuthology.orchestra.run.smithi129.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-04-28T12:34:12.910 INFO:teuthology.orchestra.run.smithi129.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-04-28T12:34:13.010 INFO:teuthology.orchestra.run.smithi129.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-28T12:34:13.085 INFO:teuthology.orchestra.run.smithi129.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-28T12:34:13.152 INFO:teuthology.orchestra.run.smithi129.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-28T12:34:13.236 INFO:teuthology.orchestra.run.smithi129.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 424 kB/s | 35 kB 00:00 2024-04-28T12:34:13.250 INFO:teuthology.orchestra.run.smithi106.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 497 kB/s | 3.1 MB 00:06 2024-04-28T12:34:13.294 INFO:teuthology.orchestra.run.smithi129.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 3.5 MB/s | 206 kB 00:00 2024-04-28T12:34:13.403 INFO:teuthology.orchestra.run.smithi129.stdout:(121/152): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-28T12:34:13.470 INFO:teuthology.orchestra.run.smithi129.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-04-28T12:34:13.529 INFO:teuthology.orchestra.run.smithi129.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-04-28T12:34:13.587 INFO:teuthology.orchestra.run.smithi129.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-28T12:34:13.654 INFO:teuthology.orchestra.run.smithi129.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 347 kB/s | 23 kB 00:00 2024-04-28T12:34:13.722 INFO:teuthology.orchestra.run.smithi129.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-04-28T12:34:13.805 INFO:teuthology.orchestra.run.smithi129.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-04-28T12:34:13.914 INFO:teuthology.orchestra.run.smithi129.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-28T12:34:13.981 INFO:teuthology.orchestra.run.smithi129.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 694 kB/s | 46 kB 00:00 2024-04-28T12:34:14.072 INFO:teuthology.orchestra.run.smithi129.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 859 kB/s | 79 kB 00:00 2024-04-28T12:34:14.139 INFO:teuthology.orchestra.run.smithi129.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 868 kB/s | 58 kB 00:00 2024-04-28T12:34:14.215 INFO:teuthology.orchestra.run.smithi129.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-04-28T12:34:14.298 INFO:teuthology.orchestra.run.smithi129.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-04-28T12:34:14.373 INFO:teuthology.orchestra.run.smithi129.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-28T12:34:14.457 INFO:teuthology.orchestra.run.smithi129.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-04-28T12:34:14.524 INFO:teuthology.orchestra.run.smithi129.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 462 kB/s | 31 kB 00:00 2024-04-28T12:34:14.616 INFO:teuthology.orchestra.run.smithi129.stdout:(137/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-28T12:34:14.708 INFO:teuthology.orchestra.run.smithi129.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 644 kB/s | 59 kB 00:00 2024-04-28T12:34:14.808 INFO:teuthology.orchestra.run.smithi129.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 358 kB/s | 36 kB 00:00 2024-04-28T12:34:14.900 INFO:teuthology.orchestra.run.smithi129.stdout:(140/152): python3-typing-extensions-4.4.0-2.el 557 kB/s | 51 kB 00:00 2024-04-28T12:34:15.000 INFO:teuthology.orchestra.run.smithi129.stdout:(141/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-28T12:34:15.084 INFO:teuthology.orchestra.run.smithi129.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-28T12:34:15.193 INFO:teuthology.orchestra.run.smithi129.stdout:(143/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-28T12:34:15.360 INFO:teuthology.orchestra.run.smithi129.stdout:(144/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.1 MB/s | 16 MB 00:07 2024-04-28T12:34:15.385 INFO:teuthology.orchestra.run.smithi129.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 104 kB/s | 20 kB 00:00 2024-04-28T12:34:15.510 INFO:teuthology.orchestra.run.smithi129.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-28T12:34:15.919 INFO:teuthology.orchestra.run.smithi129.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 343 kB/s | 192 kB 00:00 2024-04-28T12:34:20.766 INFO:teuthology.orchestra.run.smithi106.stdout:(152/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 490 kB/s | 49 MB 01:42 2024-04-28T12:34:20.772 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-28T12:34:20.772 INFO:teuthology.orchestra.run.smithi106.stdout:Total 1.5 MB/s | 222 MB 02:28 2024-04-28T12:34:23.631 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-04-28T12:34:23.873 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-04-28T12:34:23.874 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-04-28T12:34:25.118 INFO:teuthology.orchestra.run.smithi129.stdout:(148/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 350 kB/s | 3.1 MB 00:09 2024-04-28T12:34:25.615 INFO:teuthology.orchestra.run.smithi171.stdout:(152/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 430 kB/s | 49 MB 01:56 2024-04-28T12:34:25.620 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-04-28T12:34:25.620 INFO:teuthology.orchestra.run.smithi171.stdout:Total 1.4 MB/s | 222 MB 02:39 2024-04-28T12:34:26.118 INFO:teuthology.orchestra.run.smithi129.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 328 kB/s | 3.4 MB 00:10 2024-04-28T12:34:26.152 INFO:teuthology.orchestra.run.smithi129.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-04-28T12:34:26.611 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-04-28T12:34:26.612 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-04-28T12:34:26.661 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-04-28T12:34:26.808 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-04-28T12:34:26.808 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-04-28T12:34:27.144 INFO:teuthology.orchestra.run.smithi129.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-04-28T12:34:28.665 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-04-28T12:34:28.666 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-04-28T12:34:32.881 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-04-28T12:34:33.006 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-28T12:34:33.085 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-28T12:34:33.185 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-28T12:34:33.242 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-28T12:34:33.274 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-28T12:34:33.409 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-28T12:34:33.842 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-28T12:34:33.955 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:34:34.137 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:34:34.173 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:34:34.305 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:34:34.350 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:34:34.703 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-04-28T12:34:34.729 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-28T12:34:34.795 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-28T12:34:34.810 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-28T12:34:34.827 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:34:34.849 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-28T12:34:34.902 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-28T12:34:34.938 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-28T12:34:34.962 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-28T12:34:34.980 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:34:35.004 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-28T12:34:35.012 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:34:35.109 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:34:35.137 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:34:35.203 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:34:35.404 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-28T12:34:35.463 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:34:35.584 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:34:35.614 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:34:35.633 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:34:35.680 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:34:35.705 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:34:35.743 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:34:35.872 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-28T12:34:35.894 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-28T12:34:35.948 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-28T12:34:35.976 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:34:35.988 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-28T12:34:36.030 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-28T12:34:36.078 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-28T12:34:36.080 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:34:36.111 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:34:36.173 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:34:36.203 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:34:36.217 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-28T12:34:36.225 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:34:36.267 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-28T12:34:36.331 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-28T12:34:36.380 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-28T12:34:36.419 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-28T12:34:36.428 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:34:36.461 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:34:36.538 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-28T12:34:36.576 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-28T12:34:36.606 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-28T12:34:36.638 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-28T12:34:36.673 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-28T12:34:36.694 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-28T12:34:36.710 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-28T12:34:36.754 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-28T12:34:36.800 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-28T12:34:36.827 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-28T12:34:36.832 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-28T12:34:36.869 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-28T12:34:36.905 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-28T12:34:36.989 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-28T12:34:37.014 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-28T12:34:37.076 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-28T12:34:37.114 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-28T12:34:37.146 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-28T12:34:37.172 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-28T12:34:37.193 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-28T12:34:37.305 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-28T12:34:37.345 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-28T12:34:37.381 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-28T12:34:37.403 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-28T12:34:37.411 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-28T12:34:37.454 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-28T12:34:37.531 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-28T12:34:37.564 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-28T12:34:37.608 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-28T12:34:37.636 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-28T12:34:37.707 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-28T12:34:37.760 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-28T12:34:37.836 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-28T12:34:37.980 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-28T12:34:38.038 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-28T12:34:38.078 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-28T12:34:38.199 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-28T12:34:38.237 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-28T12:34:38.284 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-28T12:34:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-28T12:34:38.564 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-28T12:34:38.650 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-28T12:34:38.659 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-28T12:34:38.678 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-28T12:34:38.721 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-28T12:34:38.760 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-28T12:34:38.801 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-28T12:34:38.956 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-28T12:34:38.968 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-28T12:34:39.029 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-28T12:34:39.045 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-28T12:34:39.084 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-28T12:34:39.135 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-28T12:34:39.157 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-28T12:34:39.187 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-28T12:34:39.224 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-28T12:34:39.230 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-28T12:34:39.271 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-28T12:34:39.282 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-28T12:34:39.307 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-28T12:34:39.322 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-28T12:34:39.350 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-28T12:34:39.390 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-28T12:34:39.392 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-28T12:34:39.429 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-28T12:34:39.489 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-28T12:34:39.537 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-28T12:34:39.548 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-28T12:34:39.589 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-28T12:34:39.619 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-28T12:34:39.661 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-28T12:34:39.746 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-28T12:34:39.757 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-28T12:34:39.794 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-28T12:34:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-28T12:34:39.834 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-28T12:34:39.839 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-28T12:34:39.860 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-28T12:34:39.884 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-28T12:34:39.895 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-28T12:34:39.968 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-28T12:34:40.050 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-28T12:34:40.089 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-28T12:34:40.096 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-28T12:34:40.134 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-28T12:34:40.144 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-28T12:34:40.161 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-28T12:34:40.178 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-28T12:34:40.196 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-28T12:34:40.211 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-28T12:34:40.235 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-28T12:34:40.267 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-28T12:34:40.297 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-28T12:34:40.331 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-28T12:34:40.365 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-28T12:34:40.384 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-28T12:34:40.406 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-28T12:34:40.441 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-28T12:34:40.536 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-28T12:34:40.560 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-28T12:34:40.588 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-28T12:34:40.601 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-28T12:34:40.618 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-28T12:34:40.640 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-28T12:34:40.666 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-28T12:34:40.686 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-28T12:34:40.745 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-28T12:34:40.766 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-28T12:34:40.797 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-28T12:34:40.820 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-28T12:34:40.857 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-28T12:34:40.909 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-28T12:34:40.959 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-28T12:34:41.036 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-28T12:34:41.083 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-28T12:34:41.224 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-28T12:34:41.280 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-28T12:34:41.322 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-28T12:34:41.392 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-28T12:34:41.559 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-28T12:34:41.595 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-28T12:34:41.609 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-28T12:34:41.643 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-28T12:34:41.661 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-28T12:34:41.744 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-28T12:34:41.801 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-28T12:34:41.889 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-28T12:34:41.928 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-28T12:34:42.107 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-28T12:34:42.134 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-28T12:34:42.212 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-28T12:34:42.589 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-28T12:34:42.660 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-28T12:34:42.694 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-28T12:34:42.806 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-28T12:34:42.874 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-28T12:34:42.879 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-28T12:34:42.976 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-28T12:34:42.991 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-28T12:34:43.019 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-28T12:34:43.250 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-28T12:34:43.275 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-28T12:34:43.370 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-28T12:34:44.195 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-28T12:34:44.238 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-28T12:34:44.322 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-28T12:34:44.346 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-28T12:34:44.350 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-28T12:34:44.383 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-28T12:34:44.696 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-28T12:34:44.746 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:34:44.836 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:34:44.867 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-28T12:34:44.904 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-28T12:34:45.382 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-28T12:34:45.416 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:34:45.487 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:34:45.517 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-28T12:34:45.970 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-28T12:34:46.073 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-28T12:34:46.133 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-28T12:34:46.213 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-28T12:34:46.601 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-28T12:34:46.635 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:34:46.754 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:34:46.791 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-28T12:34:46.848 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-28T12:34:47.389 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-28T12:34:47.430 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:34:47.522 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:34:47.548 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-28T12:34:47.810 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:34:47.863 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:34:48.410 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:34:48.450 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:34:48.544 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:34:48.544 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-28T12:34:48.544 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-28T12:34:48.544 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:34:48.585 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:34:50.446 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:34:50.486 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:34:51.131 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:34:51.190 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:34:51.413 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:34:51.414 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-28T12:34:51.414 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-28T12:34:51.414 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:34:51.458 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:35:00.156 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:35:00.157 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-04-28T12:35:00.157 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-04-28T12:35:00.157 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-04-28T12:35:00.157 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-04-28T12:35:00.157 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-04-28T12:35:00.157 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-04-28T12:35:00.157 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-04-28T12:35:00.157 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:00.331 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:00.346 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:00.346 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:00.346 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-28T12:35:00.346 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:01.817 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:01.842 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:01.843 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:01.843 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-28T12:35:01.843 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:01.843 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:01.843 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:02.077 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:02.099 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:02.099 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:02.099 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-28T12:35:02.099 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:02.099 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:02.099 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:02.544 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:02.566 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:02.566 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:02.567 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-28T12:35:02.567 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:02.567 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:02.567 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:02.674 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-28T12:35:02.723 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-28T12:35:02.779 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-28T12:35:02.964 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-28T12:35:03.019 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-28T12:35:03.099 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-28T12:35:03.186 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:35:03.316 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:35:03.362 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-28T12:35:03.402 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-28T12:35:03.429 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-28T12:35:03.822 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:03.878 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:03.910 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-28T12:35:03.980 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-28T12:35:04.010 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:04.388 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:35:04.388 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2024-04-28T12:35:04.388 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2024-04-28T12:35:04.388 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2024-04-28T12:35:04.388 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2024-04-28T12:35:04.388 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2024-04-28T12:35:04.388 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2024-04-28T12:35:04.388 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2024-04-28T12:35:04.388 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:04.630 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:04.675 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:04.675 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:04.676 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-28T12:35:04.676 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:05.049 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:05.078 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:06.042 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:06.070 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:06.173 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:06.315 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:06.340 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-28T12:35:06.372 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:06.393 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:06.394 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:06.394 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-28T12:35:06.394 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:06.394 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:06.394 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:06.394 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:06.394 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:06.394 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-28T12:35:06.395 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:06.395 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:06.395 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:06.439 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:06.450 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:06.548 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-28T12:35:06.596 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-28T12:35:06.649 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:06.718 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:06.718 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:06.718 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-28T12:35:06.718 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:06.718 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:06.718 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:07.200 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:07.269 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:07.270 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:07.270 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-28T12:35:07.270 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:07.270 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:07.270 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:07.408 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-28T12:35:07.471 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-28T12:35:07.521 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-28T12:35:07.548 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-28T12:35:07.550 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:07.572 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:07.572 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:07.572 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-28T12:35:07.572 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:07.572 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:07.572 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:07.618 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:07.639 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:07.639 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:07.639 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-28T12:35:07.640 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:07.685 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:07.707 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:07.707 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:07.707 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-28T12:35:07.707 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:07.707 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:07.707 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:07.776 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-28T12:35:07.837 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-28T12:35:07.914 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-28T12:35:08.011 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:08.022 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:35:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:08.033 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:08.033 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-28T12:35:08.033 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:08.033 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:08.033 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:08.210 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:35:08.254 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-28T12:35:08.294 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-28T12:35:08.327 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-28T12:35:08.968 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:09.014 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:09.070 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-28T12:35:09.179 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-28T12:35:09.216 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:10.509 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:10.551 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:11.649 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:11.685 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:11.829 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:12.007 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-28T12:35:12.037 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:12.107 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:12.108 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:12.108 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-28T12:35:12.108 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:12.108 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:12.108 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:12.163 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:12.199 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:12.295 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-28T12:35:12.367 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-28T12:35:12.648 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-28T12:35:12.689 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-28T12:35:12.731 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-28T12:35:12.762 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-28T12:35:12.790 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-28T12:35:12.978 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-28T12:35:13.036 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-28T12:35:13.065 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-28T12:35:13.364 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-28T12:35:13.399 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:13.469 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:13.469 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:13.469 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-28T12:35:13.469 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:13.470 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:13.470 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:13.524 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:13.608 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:13.608 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:13.608 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-28T12:35:13.608 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:13.679 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:35:13.692 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:13.721 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:35:13.741 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:35:13.796 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:13.796 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:13.796 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-28T12:35:13.796 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:13.796 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:13.796 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:14.175 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:14.252 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:14.252 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:14.252 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-28T12:35:14.252 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:14.252 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:14.252 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:14.330 INFO:teuthology.orchestra.run.smithi129.stdout:(152/152): ceph-test-19.0.0-3306.gbc612ffb.el9. 348 kB/s | 49 MB 02:24 2024-04-28T12:35:14.333 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-04-28T12:35:14.334 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.1 MB/s | 222 MB 03:22 2024-04-28T12:35:14.583 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-28T12:35:14.662 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-28T12:35:14.703 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-28T12:35:14.784 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-28T12:35:14.816 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-28T12:35:14.816 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:35:14.883 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:35:14.907 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:35:14.907 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:35:14.985 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:35:15.073 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:35:15.371 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-04-28T12:35:15.514 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-04-28T12:35:15.514 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-04-28T12:35:17.284 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-04-28T12:35:17.285 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-04-28T12:35:19.309 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-28T12:35:19.366 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-28T12:35:19.459 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-28T12:35:19.504 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-28T12:35:19.553 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-28T12:35:19.873 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-28T12:35:19.946 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-28T12:35:19.974 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-28T12:35:20.991 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-28T12:35:20.992 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-28T12:35:20.993 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-28T12:35:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-28T12:35:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-28T12:35:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-28T12:35:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-28T12:35:20.994 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-28T12:35:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-28T12:35:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-28T12:35:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-28T12:35:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-28T12:35:21.006 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-28T12:35:21.007 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-28T12:35:21.008 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-28T12:35:21.009 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-28T12:35:21.010 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-28T12:35:21.011 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-28T12:35:21.012 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-28T12:35:21.013 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-28T12:35:21.014 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-28T12:35:21.015 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-28T12:35:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-28T12:35:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-28T12:35:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-28T12:35:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-28T12:35:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:35:21.016 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-28T12:35:21.022 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:35:21.063 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:35:21.118 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:35:22.464 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-28T12:35:22.570 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-28T12:35:22.645 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-28T12:35:22.831 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-28T12:35:22.876 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-28T12:35:22.877 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:35:22.995 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:35:23.042 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:35:23.042 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-04-28T12:35:23.154 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.155 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-28T12:35:23.156 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.157 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-28T12:35:23.158 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.14-1.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.159 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-28T12:35:23.160 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-28T12:35:23.161 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-28T12:35:23.162 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-28T12:35:23.163 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-3.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-28T12:35:23.164 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-28T12:35:23.165 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-28T12:35:23.165 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-28T12:35:23.165 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:35:23.165 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-04-28T12:35:23.177 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:35:23.210 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-04-28T12:35:23.293 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:35:23.317 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-28T12:35:23.372 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-28T12:35:23.434 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-28T12:35:23.468 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-28T12:35:23.494 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-28T12:35:23.536 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-28T12:35:23.586 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:35:23.586 INFO:teuthology.orchestra.run.smithi106.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-28T12:35:23.586 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:23.608 DEBUG:teuthology.parallel:result is None 2024-04-28T12:35:23.874 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-28T12:35:23.986 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:35:24.107 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:35:24.137 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:35:24.228 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:35:24.258 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:35:24.409 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-28T12:35:24.464 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-28T12:35:24.500 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:35:24.586 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:35:24.609 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:35:24.671 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-28T12:35:24.693 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:35:24.718 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-28T12:35:24.902 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:35:24.935 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-28T12:35:25.042 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-28T12:35:25.129 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-28T12:35:25.167 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-28T12:35:25.202 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-28T12:35:25.266 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-28T12:35:25.318 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-28T12:35:25.362 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-28T12:35:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-28T12:35:25.430 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-28T12:35:25.509 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-28T12:35:25.578 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-28T12:35:25.609 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-28T12:35:25.653 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-28T12:35:25.729 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-28T12:35:25.763 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-28T12:35:25.806 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-28T12:35:25.835 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-28T12:35:25.906 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-28T12:35:25.983 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-28T12:35:26.065 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-28T12:35:26.207 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-28T12:35:26.260 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-28T12:35:26.786 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-28T12:35:26.863 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-28T12:35:26.893 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-28T12:35:26.936 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-28T12:35:26.974 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-28T12:35:27.016 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-28T12:35:27.186 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-28T12:35:27.235 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-28T12:35:27.298 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-28T12:35:27.370 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-28T12:35:27.402 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-28T12:35:27.439 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-28T12:35:27.485 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-28T12:35:27.521 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-28T12:35:27.564 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-28T12:35:27.607 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-28T12:35:27.742 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-28T12:35:27.804 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-28T12:35:27.834 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-28T12:35:27.875 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-28T12:35:27.969 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-28T12:35:28.009 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-28T12:35:28.049 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-28T12:35:28.074 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-28T12:35:28.099 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-28T12:35:28.174 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-28T12:35:28.309 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-28T12:35:28.356 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-28T12:35:28.393 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-28T12:35:28.435 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-28T12:35:28.474 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-28T12:35:28.506 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-28T12:35:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-28T12:35:28.570 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-28T12:35:28.604 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-28T12:35:28.645 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-28T12:35:28.680 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-28T12:35:28.767 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-28T12:35:28.818 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-28T12:35:28.858 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-28T12:35:28.905 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-28T12:35:29.003 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-28T12:35:29.051 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-28T12:35:29.549 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-28T12:35:29.549 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-28T12:35:29.549 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:35:29.550 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-28T12:35:29.551 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-28T12:35:29.552 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-28T12:35:29.552 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-28T12:35:29.552 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-28T12:35:29.552 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-28T12:35:29.552 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-28T12:35:29.552 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-28T12:35:29.552 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-28T12:35:29.552 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-28T12:35:29.565 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-28T12:35:29.565 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-28T12:35:29.565 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-28T12:35:29.565 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-28T12:35:29.565 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-28T12:35:29.565 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-28T12:35:29.565 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-28T12:35:29.565 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-28T12:35:29.565 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-28T12:35:29.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-28T12:35:29.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-28T12:35:29.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-28T12:35:29.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-28T12:35:29.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-28T12:35:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-28T12:35:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-28T12:35:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-28T12:35:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-28T12:35:29.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-28T12:35:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-28T12:35:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-28T12:35:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-28T12:35:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-28T12:35:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-28T12:35:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-28T12:35:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-28T12:35:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-28T12:35:29.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-28T12:35:29.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-28T12:35:29.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-28T12:35:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-28T12:35:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:35:29.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-28T12:35:29.777 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-28T12:35:29.823 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-28T12:35:29.857 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-28T12:35:29.965 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-28T12:35:30.015 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-28T12:35:30.104 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-28T12:35:30.150 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-28T12:35:30.328 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-28T12:35:30.357 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-28T12:35:30.451 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-28T12:35:31.104 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-28T12:35:31.221 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-28T12:35:32.026 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:35:32.026 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:32.026 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2024-04-28T12:35:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-28T12:35:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-28T12:35:32.027 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-04-28T12:35:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-28T12:35:32.027 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2024-04-28T12:35:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2024-04-28T12:35:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-28T12:35:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.028 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.029 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.029 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.029 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.029 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.029 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.029 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.029 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.030 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.030 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.030 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.030 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.030 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.030 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.030 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.031 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.031 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.031 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.031 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.031 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.031 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:35:32.032 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-28T12:35:32.032 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2024-04-28T12:35:32.032 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-28T12:35:32.032 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-28T12:35:32.032 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-28T12:35:32.032 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-28T12:35:32.033 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-28T12:35:32.033 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-28T12:35:32.033 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-28T12:35:32.033 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-28T12:35:32.033 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-28T12:35:32.034 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.035 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.035 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.035 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-28T12:35:32.035 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-28T12:35:32.035 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-28T12:35:32.035 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-28T12:35:32.036 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-28T12:35:32.036 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.036 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.036 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-28T12:35:32.036 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.036 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-28T12:35:32.037 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-28T12:35:32.037 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-28T12:35:32.037 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-28T12:35:32.037 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-28T12:35:32.037 INFO:teuthology.orchestra.run.smithi106.stdout: lua-5.4.4-4.el9.x86_64 2024-04-28T12:35:32.037 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-28T12:35:32.038 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-28T12:35:32.038 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-28T12:35:32.038 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-28T12:35:32.038 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-28T12:35:32.038 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-28T12:35:32.038 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.14-1.el9.x86_64 2024-04-28T12:35:32.039 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-28T12:35:32.039 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-28T12:35:32.039 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-28T12:35:32.039 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-28T12:35:32.039 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-28T12:35:32.039 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-28T12:35:32.040 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-28T12:35:32.040 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-28T12:35:32.040 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-28T12:35:32.040 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-28T12:35:32.040 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-28T12:35:32.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-28T12:35:32.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-28T12:35:32.041 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-28T12:35:32.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-28T12:35:32.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-28T12:35:32.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-28T12:35:32.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-28T12:35:32.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-28T12:35:32.042 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-28T12:35:32.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-28T12:35:32.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-28T12:35:32.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-28T12:35:32.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-28T12:35:32.043 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-28T12:35:32.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-28T12:35:32.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-28T12:35:32.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-28T12:35:32.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-28T12:35:32.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-28T12:35:32.044 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-28T12:35:32.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-28T12:35:32.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-28T12:35:32.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-28T12:35:32.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-28T12:35:32.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-28T12:35:32.045 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-28T12:35:32.046 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-28T12:35:32.046 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-28T12:35:32.046 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-28T12:35:32.046 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2024-04-28T12:35:32.046 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-28T12:35:32.047 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-28T12:35:32.047 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-28T12:35:32.047 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-28T12:35:32.047 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-28T12:35:32.048 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-28T12:35:32.048 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-28T12:35:32.048 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-28T12:35:32.048 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-28T12:35:32.048 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.048 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.049 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-28T12:35:32.049 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-28T12:35:32.049 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-28T12:35:32.049 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.049 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-28T12:35:32.049 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-28T12:35:32.049 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.050 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:35:32.051 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-28T12:35:32.457 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-28T12:35:32.543 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-28T12:35:32.571 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-28T12:35:32.576 DEBUG:teuthology.parallel:result is None 2024-04-28T12:35:32.604 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-28T12:35:32.917 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-28T12:35:32.959 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:35:33.049 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-28T12:35:33.080 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-28T12:35:33.116 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-28T12:35:33.613 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-28T12:35:33.654 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:35:33.716 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-28T12:35:33.739 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-28T12:35:36.022 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:35:36.068 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:35:36.614 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-28T12:35:36.655 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:35:36.733 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-28T12:35:36.733 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-28T12:35:36.733 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-28T12:35:36.734 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:36.764 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:35:47.977 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-28T12:35:47.977 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-04-28T12:35:47.977 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-04-28T12:35:47.977 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-04-28T12:35:47.977 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-04-28T12:35:47.977 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-04-28T12:35:47.977 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-04-28T12:35:47.977 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-04-28T12:35:47.977 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:48.150 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:48.165 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-28T12:35:48.165 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:48.165 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-28T12:35:48.165 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:49.636 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:49.660 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-28T12:35:49.660 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:49.660 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-28T12:35:49.660 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:49.661 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-28T12:35:49.661 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:49.887 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:49.908 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-28T12:35:49.909 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:49.909 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-28T12:35:49.909 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:49.909 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-28T12:35:49.909 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:50.354 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:50.376 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-28T12:35:50.376 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:50.376 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-28T12:35:50.376 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:50.376 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-28T12:35:50.376 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:50.492 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-28T12:35:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-28T12:35:50.613 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-28T12:35:50.797 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-28T12:35:50.846 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-28T12:35:50.932 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-28T12:35:51.028 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:35:51.158 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:35:51.196 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-28T12:35:51.228 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-28T12:35:51.256 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-28T12:35:51.632 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:51.662 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-28T12:35:51.703 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-28T12:35:51.773 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-28T12:35:51.802 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-28T12:35:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:53.841 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-28T12:35:53.863 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:53.964 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-28T12:35:54.116 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-28T12:35:54.140 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:54.162 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-28T12:35:54.163 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:54.163 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-28T12:35:54.163 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:54.163 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-28T12:35:54.163 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:54.199 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:54.209 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-28T12:35:54.307 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-28T12:35:54.356 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-28T12:35:55.282 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-28T12:35:55.309 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:55.331 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-28T12:35:55.332 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:55.332 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-28T12:35:55.332 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:55.332 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-28T12:35:55.332 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:55.377 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:55.398 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-28T12:35:55.398 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:55.399 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-28T12:35:55.399 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:55.444 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:55.466 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-28T12:35:55.466 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:55.466 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-28T12:35:55.466 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:55.466 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-28T12:35:55.466 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:35:55.770 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:55.792 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-28T12:35:55.792 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-28T12:35:55.792 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-28T12:35:55.792 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:55.792 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-28T12:35:55.792 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:36:00.390 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-28T12:36:00.432 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-28T12:36:00.472 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-28T12:36:00.504 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-28T12:36:00.532 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-28T12:36:00.717 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-28T12:36:00.761 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-28T12:36:00.791 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-28T12:36:01.439 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:36:01.480 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:36:01.499 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-28T12:36:02.327 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-28T12:36:02.418 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-28T12:36:02.458 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-28T12:36:02.540 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-28T12:36:02.572 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-28T12:36:02.572 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:36:02.639 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-28T12:36:02.663 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:36:02.664 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:36:02.756 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-28T12:36:02.822 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:36:08.289 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:36:08.289 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-28T12:36:08.289 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-28T12:36:08.289 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-28T12:36:08.289 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-28T12:36:08.289 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-28T12:36:08.289 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-28T12:36:08.289 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-28T12:36:08.289 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-28T12:36:08.290 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-28T12:36:08.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-28T12:36:08.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-28T12:36:08.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-28T12:36:08.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-28T12:36:08.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-28T12:36:08.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-28T12:36:08.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-28T12:36:08.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-28T12:36:08.291 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-28T12:36:08.303 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-28T12:36:08.303 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-28T12:36:08.303 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-28T12:36:08.303 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-28T12:36:08.303 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-28T12:36:08.303 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-28T12:36:08.304 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-28T12:36:08.305 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-28T12:36:08.306 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-28T12:36:08.307 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-28T12:36:08.308 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-28T12:36:08.309 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-28T12:36:08.310 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-28T12:36:08.311 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-28T12:36:08.312 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-28T12:36:08.313 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-28T12:36:08.313 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-28T12:36:08.313 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-28T12:36:08.313 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-28T12:36:08.313 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-28T12:36:08.313 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-28T12:36:08.313 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-28T12:36:10.463 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: automake-1.16.2-8.el9.noarch 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: bison-3.7.4-5.el9.x86_64 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.464 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.465 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: flex-2.6.4-9.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-28T12:36:10.466 INFO:teuthology.orchestra.run.smithi129.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-28T12:36:10.467 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-28T12:36:10.468 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-13.14-1.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.469 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-28T12:36:10.470 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-28T12:36:10.471 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-28T12:36:10.472 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-28T12:36:10.473 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-3.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-28T12:36:10.474 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-28T12:36:10.475 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-28T12:36:10.475 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-04-28T12:36:10.475 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-04-28T12:36:10.892 DEBUG:teuthology.parallel:result is None 2024-04-28T12:36:10.893 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:36:11.119 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-28T12:36:11.192 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:11.192 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:11.192 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-28T12:36:11.194 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:36:11.337 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-28T12:36:11.358 INFO:teuthology.orchestra.run.smithi129.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:11.359 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:11.359 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-28T12:36:11.360 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:36:11.500 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-28T12:36:11.526 INFO:teuthology.orchestra.run.smithi171.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:11.526 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-28T12:36:11.526 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-28T12:36:11.528 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-28T12:36:11.528 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:36:11.528 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-28T12:36:11.614 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:36:11.614 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-28T12:36:11.640 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:36:11.640 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-28T12:36:11.672 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-28T12:36:11.672 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:36:11.672 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-04-28T12:36:11.759 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-28T12:36:11.841 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:36:11.841 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-04-28T12:36:11.864 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-28T12:36:11.927 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:36:11.928 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-04-28T12:36:11.956 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-28T12:36:12.023 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-28T12:36:12.023 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:36:12.023 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-28T12:36:12.114 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-28T12:36:12.243 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:36:12.243 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-28T12:36:12.267 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-28T12:36:12.330 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:36:12.330 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-28T12:36:12.358 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-28T12:36:12.426 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-28T12:36:12.427 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:36:12.427 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-04-28T12:36:12.515 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-28T12:36:12.598 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:36:12.598 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-04-28T12:36:12.622 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-28T12:36:12.685 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:36:12.685 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-04-28T12:36:12.713 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-28T12:36:12.780 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-28T12:36:12.881 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'subvols': {'create': 2}} 2024-04-28T12:36:12.881 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:36:12.882 INFO:tasks.cephadm:Cluster fsid is e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:36:12.882 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-28T12:36:12.882 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.106', 'mon.b': '172.21.15.129', 'mon.c': '172.21.15.171'} 2024-04-28T12:36:12.882 INFO:tasks.cephadm:First mon is mon.a on smithi106 2024-04-28T12:36:12.882 INFO:tasks.cephadm:First mgr is x 2024-04-28T12:36:12.882 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-28T12:36:12.883 DEBUG:teuthology.orchestra.run.smithi106:> sudo hostname $(hostname -s) 2024-04-28T12:36:12.975 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname $(hostname -s) 2024-04-28T12:36:12.999 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-04-28T12:36:13.027 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-28T12:36:13.027 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:36:13.246 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78918/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3306-gbc612ffb', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3306.gbc612ffb'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 13:02:42.256491', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240426.111959', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-28T12:36:13.389 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240426.111959, sha1 bc612ffb9c1d5bc27536bc917a705262a36d4387 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:36:13.390 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm 2024-04-28T12:36:13.391 DEBUG:teuthology.orchestra.run.smithi106:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-28T12:36:15.067 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 28 12:36 /home/ubuntu/cephtest/cephadm 2024-04-28T12:36:15.067 DEBUG:teuthology.orchestra.run.smithi129:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-28T12:36:16.567 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 28 12:36 /home/ubuntu/cephtest/cephadm 2024-04-28T12:36:16.567 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-28T12:36:17.888 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 781514 Apr 28 12:36 /home/ubuntu/cephtest/cephadm 2024-04-28T12:36:17.888 DEBUG:teuthology.orchestra.run.smithi106:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-28T12:36:17.949 DEBUG:teuthology.orchestra.run.smithi129:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-28T12:36:17.964 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-28T12:36:18.000 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 on all hosts... 2024-04-28T12:36:18.000 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-28T12:36:18.007 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-28T12:36:18.009 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 pull 2024-04-28T12:36:19.564 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-28T12:36:19.881 INFO:teuthology.orchestra.run.smithi106.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-28T12:36:20.028 INFO:teuthology.orchestra.run.smithi129.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-28T12:37:30.683 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-04-28T12:37:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:37:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-28T12:37:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-04-28T12:37:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-28T12:37:30.684 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-04-28T12:37:30.684 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-04-28T12:37:36.035 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:37:36.035 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:37:36.035 INFO:teuthology.orchestra.run.smithi106.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-28T12:37:36.036 INFO:teuthology.orchestra.run.smithi106.stdout: "repo_digests": [ 2024-04-28T12:37:36.036 INFO:teuthology.orchestra.run.smithi106.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-28T12:37:36.036 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-28T12:37:36.036 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:39:36.432 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-04-28T12:39:36.433 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:39:36.433 INFO:teuthology.orchestra.run.smithi129.stdout: "image_id": "55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07", 2024-04-28T12:39:36.433 INFO:teuthology.orchestra.run.smithi129.stdout: "repo_digests": [ 2024-04-28T12:39:36.433 INFO:teuthology.orchestra.run.smithi129.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2" 2024-04-28T12:39:36.433 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-04-28T12:39:36.433 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-04-28T12:39:36.457 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph 2024-04-28T12:39:36.551 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph 2024-04-28T12:39:36.578 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-04-28T12:39:36.611 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /etc/ceph 2024-04-28T12:39:36.698 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /etc/ceph 2024-04-28T12:39:36.722 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-04-28T12:39:36.749 INFO:tasks.cephadm:Writing seed config... 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-28T12:39:36.751 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-28T12:39:36.752 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-28T12:39:36.753 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-28T12:39:36.754 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-28T12:39:36.754 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-28T12:39:36.754 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-28T12:39:36.754 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-28T12:39:36.754 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-28T12:39:36.754 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-28T12:39:36.755 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:39:36.755 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-28T12:39:36.810 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e5e35dfa-055b-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-28T12:39:36.810 DEBUG:teuthology.orchestra.run.smithi106:mon.a> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.a.service 2024-04-28T12:39:36.858 DEBUG:teuthology.orchestra.run.smithi106:mgr.x> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.x.service 2024-04-28T12:39:36.904 INFO:tasks.cephadm:Bootstrapping... 2024-04-28T12:39:36.905 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 -v bootstrap --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.106 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-28T12:39:37.183 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-28T12:39:37.184 INFO:teuthology.orchestra.run.smithi106.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387', '-v', 'bootstrap', '--fsid', 'e5e35dfa-055b-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.106', '--skip-admin-label'] 2024-04-28T12:39:37.184 INFO:teuthology.orchestra.run.smithi106.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-28T12:39:37.184 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying podman|docker is present... 2024-04-28T12:39:37.239 INFO:teuthology.orchestra.run.smithi106.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-28T12:39:37.239 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying lvm2 is present... 2024-04-28T12:39:37.239 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying time synchronization is in place... 2024-04-28T12:39:37.254 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-28T12:39:37.254 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-28T12:39:37.267 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-28T12:39:37.268 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout inactive 2024-04-28T12:39:37.287 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout enabled 2024-04-28T12:39:37.307 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout active 2024-04-28T12:39:37.307 INFO:teuthology.orchestra.run.smithi106.stdout:Unit chronyd.service is enabled and running 2024-04-28T12:39:37.307 INFO:teuthology.orchestra.run.smithi106.stdout:Repeating the final host check... 2024-04-28T12:39:37.365 INFO:teuthology.orchestra.run.smithi106.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-28T12:39:37.365 INFO:teuthology.orchestra.run.smithi106.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-28T12:39:37.366 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl is present 2024-04-28T12:39:37.366 INFO:teuthology.orchestra.run.smithi106.stdout:lvcreate is present 2024-04-28T12:39:37.380 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-28T12:39:37.380 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-28T12:39:37.394 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-28T12:39:37.394 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout inactive 2024-04-28T12:39:37.410 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout enabled 2024-04-28T12:39:37.424 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout active 2024-04-28T12:39:37.424 INFO:teuthology.orchestra.run.smithi106.stdout:Unit chronyd.service is enabled and running 2024-04-28T12:39:37.424 INFO:teuthology.orchestra.run.smithi106.stdout:Host looks OK 2024-04-28T12:39:37.424 INFO:teuthology.orchestra.run.smithi106.stdout:Cluster fsid: e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:37.425 INFO:teuthology.orchestra.run.smithi106.stdout:Acquiring lock 140586459856032 on /run/cephadm/e5e35dfa-055b-11ef-bc93-c7b262605968.lock 2024-04-28T12:39:37.425 INFO:teuthology.orchestra.run.smithi106.stdout:Lock 140586459856032 acquired on /run/cephadm/e5e35dfa-055b-11ef-bc93-c7b262605968.lock 2024-04-28T12:39:37.425 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying IP 172.21.15.106 port 3300 ... 2024-04-28T12:39:37.427 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying IP 172.21.15.106 port 6789 ... 2024-04-28T12:39:37.427 INFO:teuthology.orchestra.run.smithi106.stdout:Base mon IP(s) is [172.21.15.106:3300, 172.21.15.106:6789], mon addrv is [v2:172.21.15.106:3300,v1:172.21.15.106:6789] 2024-04-28T12:39:37.436 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.106 metric 100 2024-04-28T12:39:37.436 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.106 metric 100 2024-04-28T12:39:37.445 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-28T12:39:37.445 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-28T12:39:37.445 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1758sec hoplimit 64 pref medium 2024-04-28T12:39:37.453 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-28T12:39:37.454 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-28T12:39:37.454 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-28T12:39:37.454 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-28T12:39:37.454 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-28T12:39:37.454 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-28T12:39:37.455 INFO:teuthology.orchestra.run.smithi106.stdout:Mon IP `172.21.15.106` is in CIDR network `172.21.0.0/20` 2024-04-28T12:39:37.455 INFO:teuthology.orchestra.run.smithi106.stdout:Mon IP `172.21.15.106` is in CIDR network `172.21.0.0/20` 2024-04-28T12:39:37.455 INFO:teuthology.orchestra.run.smithi106.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-28T12:39:37.457 INFO:teuthology.orchestra.run.smithi106.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-28T12:39:37.458 INFO:teuthology.orchestra.run.smithi106.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-28T12:39:38.315 INFO:teuthology.orchestra.run.smithi106.stdout:/bin/podman: stdout 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-28T12:39:38.316 INFO:teuthology.orchestra.run.smithi106.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387... 2024-04-28T12:39:38.316 INFO:teuthology.orchestra.run.smithi106.stdout:/bin/podman: stderr Getting image source signatures 2024-04-28T12:39:38.316 INFO:teuthology.orchestra.run.smithi106.stdout:/bin/podman: stderr Copying blob sha256:f340d36a00bce3feed0ad509aeafc9092c1c625bf553b8a613c380a79149b87e 2024-04-28T12:39:38.316 INFO:teuthology.orchestra.run.smithi106.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-28T12:39:38.316 INFO:teuthology.orchestra.run.smithi106.stdout:/bin/podman: stderr Copying config sha256:55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 2024-04-28T12:39:38.316 INFO:teuthology.orchestra.run.smithi106.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-28T12:39:39.811 INFO:teuthology.orchestra.run.smithi106.stdout:ceph: stdout ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-28T12:39:39.811 INFO:teuthology.orchestra.run.smithi106.stdout:Ceph version: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev) 2024-04-28T12:39:39.811 INFO:teuthology.orchestra.run.smithi106.stdout:Extracting ceph user uid/gid from container image... 2024-04-28T12:39:41.195 INFO:teuthology.orchestra.run.smithi106.stdout:stat: stdout 167 167 2024-04-28T12:39:41.196 INFO:teuthology.orchestra.run.smithi106.stdout:Creating initial keys... 2024-04-28T12:39:42.575 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph-authtool: stdout AQCNQy5m6P3YJRAA6nOLlssNwi3zqnL5xX9Ekw== 2024-04-28T12:39:43.705 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph-authtool: stdout AQCPQy5mZgEGBRAAY2/6/IXyqz0JUUINDmMhbQ== 2024-04-28T12:39:44.756 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph-authtool: stdout AQCQQy5mPmpHCRAA8nbXQtv81zCYOfBSMX8xdQ== 2024-04-28T12:39:44.759 INFO:teuthology.orchestra.run.smithi106.stdout:Creating initial monmap... 2024-04-28T12:39:45.905 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-28T12:39:45.906 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-28T12:39:45.906 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:45.906 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-28T12:39:45.906 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool for a [v2:172.21.15.106:3300,v1:172.21.15.106:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-28T12:39:45.906 INFO:teuthology.orchestra.run.smithi106.stdout:setting min_mon_release = quincy 2024-04-28T12:39:45.906 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/monmaptool: set fsid to e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:45.907 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-28T12:39:45.907 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:39:45.907 INFO:teuthology.orchestra.run.smithi106.stdout:Creating mon... 2024-04-28T12:39:47.046 INFO:teuthology.orchestra.run.smithi106.stdout:create mon.a on 2024-04-28T12:39:47.427 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-28T12:39:47.675 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-28T12:39:47.974 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968.target. 2024-04-28T12:39:47.974 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968.target. 2024-04-28T12:39:48.374 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.a 2024-04-28T12:39:48.374 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-28T12:39:48.677 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968.target.wants/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@.service. 2024-04-28T12:39:49.315 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:49 smithi106 podman[27694]: 2024-04-28 12:39:49.306005285 +0000 UTC m=+0.362423454 container init 4af5eddd69c55db75e5b37aa8a462ce18e13b19932fe06cd9d3362b298333c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-28T12:39:49.316 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:49 smithi106 podman[27694]: 2024-04-28 12:39:49.316782829 +0000 UTC m=+0.373200999 container start 4af5eddd69c55db75e5b37aa8a462ce18e13b19932fe06cd9d3362b298333c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T12:39:49.423 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-28T12:39:49.424 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout disabled 2024-04-28T12:39:49.441 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-28T12:39:49.441 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout inactive 2024-04-28T12:39:49.441 INFO:teuthology.orchestra.run.smithi106.stdout:firewalld.service is not enabled 2024-04-28T12:39:49.441 INFO:teuthology.orchestra.run.smithi106.stdout:Not possible to enable service . firewalld.service is not available 2024-04-28T12:39:49.442 INFO:teuthology.orchestra.run.smithi106.stdout:Waiting for mon to start... 2024-04-28T12:39:49.442 INFO:teuthology.orchestra.run.smithi106.stdout:Waiting for mon... 2024-04-28T12:39:49.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:49 smithi106 bash[27694]: 4af5eddd69c55db75e5b37aa8a462ce18e13b19932fe06cd9d3362b298333c14 2024-04-28T12:39:49.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:49 smithi106 ceph-mon[27713]: mkfs e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:49.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:49 smithi106 ceph-mon[27713]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-28T12:39:49.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:49 smithi106 systemd[1]: Started Ceph mon.a for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:39:50.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-28T12:39:50.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: monmap epoch 1 2024-04-28T12:39:50.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:50.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: last_changed 2024-04-28T12:39:45.237228+0000 2024-04-28T12:39:50.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:39:50.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: min_mon_release 19 (squid) 2024-04-28T12:39:50.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: election_strategy: 1 2024-04-28T12:39:50.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:39:50.478 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: fsmap 2024-04-28T12:39:50.478 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: osdmap e1: 0 total, 0 up, 0 in 2024-04-28T12:39:50.478 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: mgrmap e1: no daemons active 2024-04-28T12:39:50.478 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:50 smithi106 ceph-mon[27713]: from='client.? 172.21.15.106:0/3240864400' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout cluster: 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout id: e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout services: 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.772125s) 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout data: 2024-04-28T12:39:50.741 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout pgs: 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.072+0000 7f0cfd200700 1 Processor -- start 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.072+0000 7f0cfd200700 1 -- start start 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.072+0000 7f0cfd200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf8112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.072+0000 7f0cfd200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8112580 con 0x7f0cf810fc50 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.074+0000 7f0cf5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf8112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.074+0000 7f0cf5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf8112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49900/0 (socket says 172.21.15.106:49900) 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.074+0000 7f0cf5e00700 1 -- 172.21.15.106:0/2788406499 learned_addr learned my addr 172.21.15.106:0/2788406499 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:50.742 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.074+0000 7f0cf5e00700 1 -- 172.21.15.106:0/2788406499 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf8112700 con 0x7f0cf810fc50 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.075+0000 7f0cf5e00700 1 --2- 172.21.15.106:0/2788406499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf8112040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80098a0 tx=0x7f0ce8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ac8732e30f9b9b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.075+0000 7f0cf4a00700 1 -- 172.21.15.106:0/2788406499 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce8004160 con 0x7f0cf810fc50 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.075+0000 7f0cf4a00700 1 -- 172.21.15.106:0/2788406499 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce8004300 con 0x7f0cf810fc50 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.075+0000 7f0cf4a00700 1 -- 172.21.15.106:0/2788406499 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce80036a0 con 0x7f0cf810fc50 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.076+0000 7f0cfd200700 1 -- 172.21.15.106:0/2788406499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 msgr2=0x7f0cf8112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.076+0000 7f0cfd200700 1 --2- 172.21.15.106:0/2788406499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf8112040 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80098a0 tx=0x7f0ce8009bb0 comp rx=0 tx=0).stop 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.076+0000 7f0cfd200700 1 -- 172.21.15.106:0/2788406499 shutdown_connections 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.076+0000 7f0cfd200700 1 --2- 172.21.15.106:0/2788406499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf8112040 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.076+0000 7f0cfd200700 1 -- 172.21.15.106:0/2788406499 >> 172.21.15.106:0/2788406499 conn(0x7f0cf807e370 msgr2=0x7f0cf8080790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.076+0000 7f0cfd200700 1 -- 172.21.15.106:0/2788406499 shutdown_connections 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.076+0000 7f0cfd200700 1 -- 172.21.15.106:0/2788406499 wait complete. 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.077+0000 7f0cfd200700 1 Processor -- start 2024-04-28T12:39:50.743 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.077+0000 7f0cfd200700 1 -- start start 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.078+0000 7f0cfd200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf81a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.078+0000 7f0cfd200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf8112bd0 con 0x7f0cf810fc50 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.078+0000 7f0cf5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf81a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.078+0000 7f0cf5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf81a29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49906/0 (socket says 172.21.15.106:49906) 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.078+0000 7f0cf5e00700 1 -- 172.21.15.106:0/3240864400 learned_addr learned my addr 172.21.15.106:0/3240864400 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.078+0000 7f0cf5e00700 1 -- 172.21.15.106:0/3240864400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf81a2f30 con 0x7f0cf810fc50 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.079+0000 7f0cf5e00700 1 --2- 172.21.15.106:0/3240864400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf81a29f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8000c00 tx=0x7f0ce8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.079+0000 7f0ceea00700 1 -- 172.21.15.106:0/3240864400 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce8003db0 con 0x7f0cf810fc50 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.079+0000 7f0ceea00700 1 -- 172.21.15.106:0/3240864400 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce8011630 con 0x7f0cf810fc50 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.079+0000 7f0cfd200700 1 -- 172.21.15.106:0/3240864400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf81a31c0 con 0x7f0cf810fc50 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.079+0000 7f0ceea00700 1 -- 172.21.15.106:0/3240864400 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ce8011950 con 0x7f0cf810fc50 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.079+0000 7f0cfd200700 1 -- 172.21.15.106:0/3240864400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf81a5eb0 con 0x7f0cf810fc50 2024-04-28T12:39:50.744 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.079+0000 7f0ceea00700 1 -- 172.21.15.106:0/3240864400 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ce801a460 con 0x7f0cf810fc50 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.079+0000 7f0ceea00700 1 -- 172.21.15.106:0/3240864400 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ce801b780 con 0x7f0cf810fc50 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.080+0000 7f0cfd200700 1 -- 172.21.15.106:0/3240864400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf8064a80 con 0x7f0cf810fc50 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.084+0000 7f0ceea00700 1 -- 172.21.15.106:0/3240864400 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0ce801ba20 con 0x7f0cf810fc50 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.163+0000 7f0cfd200700 1 -- 172.21.15.106:0/3240864400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f0cf802e770 con 0x7f0cf810fc50 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.163+0000 7f0ceea00700 1 -- 172.21.15.106:0/3240864400 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f0ce801bc00 con 0x7f0cf810fc50 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.165+0000 7f0cfd200700 1 -- 172.21.15.106:0/3240864400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 msgr2=0x7f0cf81a29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.165+0000 7f0cfd200700 1 --2- 172.21.15.106:0/3240864400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf81a29f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8000c00 tx=0x7f0ce8004750 comp rx=0 tx=0).stop 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.165+0000 7f0cfd200700 1 -- 172.21.15.106:0/3240864400 shutdown_connections 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.165+0000 7f0cfd200700 1 --2- 172.21.15.106:0/3240864400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf810fc50 0x7f0cf81a29f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.165+0000 7f0cfd200700 1 -- 172.21.15.106:0/3240864400 >> 172.21.15.106:0/3240864400 conn(0x7f0cf807e370 msgr2=0x7f0cf807f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.165+0000 7f0cfd200700 1 -- 172.21.15.106:0/3240864400 shutdown_connections 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:50.165+0000 7f0cfd200700 1 -- 172.21.15.106:0/3240864400 wait complete. 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:mon is available 2024-04-28T12:39:50.745 INFO:teuthology.orchestra.run.smithi106.stdout:Assimilating anything we can from ceph.conf... 2024-04-28T12:39:51.741 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:51 smithi106 ceph-mon[27713]: from='client.? 172.21.15.106:0/2390521761' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-28T12:39:51.741 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:51 smithi106 ceph-mon[27713]: from='client.? 172.21.15.106:0/2390521761' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-28T12:39:52.034 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:39:52.034 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout [global] 2024-04-28T12:39:52.034 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout fsid = e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:52.034 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-28T12:39:52.034 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.106:3300,v1:172.21.15.106:6789] 2024-04-28T12:39:52.034 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-28T12:39:52.034 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-28T12:39:52.034 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-28T12:39:52.034 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-28T12:39:52.035 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:39:52.035 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-28T12:39:52.035 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-28T12:39:52.035 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:39:52.040 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout [osd] 2024-04-28T12:39:52.040 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-28T12:39:52.040 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-28T12:39:52.040 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-28T12:39:52.040 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.328+0000 7f7588e00700 1 Processor -- start 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.328+0000 7f7588e00700 1 -- start start 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.329+0000 7f7588e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f7584108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.329+0000 7f7588e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75841095a0 con 0x7f7584108bb0 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.329+0000 7f7583400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f7584108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.329+0000 7f7583400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f7584108fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49922/0 (socket says 172.21.15.106:49922) 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.329+0000 7f7583400700 1 -- 172.21.15.106:0/130568019 learned_addr learned my addr 172.21.15.106:0/130568019 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.330+0000 7f7583400700 1 -- 172.21.15.106:0/130568019 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7584109db0 con 0x7f7584108bb0 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.330+0000 7f7583400700 1 --2- 172.21.15.106:0/130568019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f7584108fd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f75740098a0 tx=0x7f7574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aabce68004a3541 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.330+0000 7f7582000700 1 -- 172.21.15.106:0/130568019 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7574004160 con 0x7f7584108bb0 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.330+0000 7f7582000700 1 -- 172.21.15.106:0/130568019 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7574004300 con 0x7f7584108bb0 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.330+0000 7f7582000700 1 -- 172.21.15.106:0/130568019 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75740036a0 con 0x7f7584108bb0 2024-04-28T12:39:52.041 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.331+0000 7f7588e00700 1 -- 172.21.15.106:0/130568019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 msgr2=0x7f7584108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.331+0000 7f7588e00700 1 --2- 172.21.15.106:0/130568019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f7584108fd0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f75740098a0 tx=0x7f7574009bb0 comp rx=0 tx=0).stop 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.331+0000 7f7588e00700 1 -- 172.21.15.106:0/130568019 shutdown_connections 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.331+0000 7f7588e00700 1 --2- 172.21.15.106:0/130568019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f7584108fd0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.331+0000 7f7588e00700 1 -- 172.21.15.106:0/130568019 >> 172.21.15.106:0/130568019 conn(0x7f7584104190 msgr2=0x7f75841065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.331+0000 7f7588e00700 1 -- 172.21.15.106:0/130568019 shutdown_connections 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.332+0000 7f7588e00700 1 -- 172.21.15.106:0/130568019 wait complete. 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.332+0000 7f7588e00700 1 Processor -- start 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.333+0000 7f7588e00700 1 -- start start 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.333+0000 7f7588e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f758419e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.333+0000 7f7588e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f758410a2e0 con 0x7f7584108bb0 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.333+0000 7f7583400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f758419e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.333+0000 7f7583400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f758419e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49934/0 (socket says 172.21.15.106:49934) 2024-04-28T12:39:52.042 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.333+0000 7f7583400700 1 -- 172.21.15.106:0/2390521761 learned_addr learned my addr 172.21.15.106:0/2390521761 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.334+0000 7f7583400700 1 -- 172.21.15.106:0/2390521761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758419e9e0 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.334+0000 7f7583400700 1 --2- 172.21.15.106:0/2390521761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f758419e4a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7574000c00 tx=0x7f7574004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.334+0000 7f757be00700 1 -- 172.21.15.106:0/2390521761 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7574003db0 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.334+0000 7f757be00700 1 -- 172.21.15.106:0/2390521761 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7574011630 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.334+0000 7f7588e00700 1 -- 172.21.15.106:0/2390521761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758419ec70 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.334+0000 7f757be00700 1 -- 172.21.15.106:0/2390521761 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7574011950 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.335+0000 7f7588e00700 1 -- 172.21.15.106:0/2390521761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75841a1960 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.335+0000 7f757be00700 1 -- 172.21.15.106:0/2390521761 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f757401a460 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.335+0000 7f757be00700 1 -- 172.21.15.106:0/2390521761 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f757401b780 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.336+0000 7f7588e00700 1 -- 172.21.15.106:0/2390521761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7584064a80 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.339+0000 7f757be00700 1 -- 172.21.15.106:0/2390521761 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7574031020 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.415+0000 7f7588e00700 1 -- 172.21.15.106:0/2390521761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f758419f150 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.420+0000 7f757be00700 1 -- 172.21.15.106:0/2390521761 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+467 (secure 0 0 0) 0x7f758419f150 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.420+0000 7f757be00700 1 -- 172.21.15.106:0/2390521761 <== mon.0 v2:172.21.15.106:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f757401a8e0 con 0x7f7584108bb0 2024-04-28T12:39:52.043 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.421+0000 7f7588e00700 1 -- 172.21.15.106:0/2390521761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 msgr2=0x7f758419e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:52.044 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.421+0000 7f7588e00700 1 --2- 172.21.15.106:0/2390521761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f758419e4a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f7574000c00 tx=0x7f7574004750 comp rx=0 tx=0).stop 2024-04-28T12:39:52.044 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.421+0000 7f7588e00700 1 -- 172.21.15.106:0/2390521761 shutdown_connections 2024-04-28T12:39:52.044 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.422+0000 7f7588e00700 1 --2- 172.21.15.106:0/2390521761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7584108bb0 0x7f758419e4a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:52.044 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.422+0000 7f7588e00700 1 -- 172.21.15.106:0/2390521761 >> 172.21.15.106:0/2390521761 conn(0x7f7584104190 msgr2=0x7f758407e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:52.044 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.422+0000 7f7588e00700 1 -- 172.21.15.106:0/2390521761 shutdown_connections 2024-04-28T12:39:52.044 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:51.422+0000 7f7588e00700 1 -- 172.21.15.106:0/2390521761 wait complete. 2024-04-28T12:39:52.044 INFO:teuthology.orchestra.run.smithi106.stdout:Generating new minimal ceph.conf... 2024-04-28T12:39:53.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:52 smithi106 ceph-mon[27713]: from='client.? 172.21.15.106:0/549850206' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:39:53.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.659+0000 7f617ae00700 1 Processor -- start 2024-04-28T12:39:53.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.659+0000 7f617ae00700 1 -- start start 2024-04-28T12:39:53.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.659+0000 7f617ae00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f6174112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:53.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.659+0000 7f617ae00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6174112580 con 0x7f617410fc50 2024-04-28T12:39:53.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.660+0000 7f6173e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f6174112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:53.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.660+0000 7f6173e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f6174112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49950/0 (socket says 172.21.15.106:49950) 2024-04-28T12:39:53.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.660+0000 7f6173e00700 1 -- 172.21.15.106:0/1015504697 learned_addr learned my addr 172.21.15.106:0/1015504697 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:53.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.661+0000 7f6173e00700 1 -- 172.21.15.106:0/1015504697 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6174112700 con 0x7f617410fc50 2024-04-28T12:39:53.325 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.661+0000 7f6173e00700 1 --2- 172.21.15.106:0/1015504697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f6174112040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61640098a0 tx=0x7f6164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba2d984b2650f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:53.325 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.661+0000 7f6172a00700 1 -- 172.21.15.106:0/1015504697 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6164004160 con 0x7f617410fc50 2024-04-28T12:39:53.325 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.662+0000 7f6172a00700 1 -- 172.21.15.106:0/1015504697 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6164004300 con 0x7f617410fc50 2024-04-28T12:39:53.325 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.662+0000 7f6172a00700 1 -- 172.21.15.106:0/1015504697 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6164011af0 con 0x7f617410fc50 2024-04-28T12:39:53.326 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.662+0000 7f617ae00700 1 -- 172.21.15.106:0/1015504697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 msgr2=0x7f6174112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:53.326 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.662+0000 7f617ae00700 1 --2- 172.21.15.106:0/1015504697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f6174112040 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61640098a0 tx=0x7f6164009bb0 comp rx=0 tx=0).stop 2024-04-28T12:39:53.326 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.662+0000 7f617ae00700 1 -- 172.21.15.106:0/1015504697 shutdown_connections 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.662+0000 7f617ae00700 1 --2- 172.21.15.106:0/1015504697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f6174112040 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.662+0000 7f617ae00700 1 -- 172.21.15.106:0/1015504697 >> 172.21.15.106:0/1015504697 conn(0x7f617407e370 msgr2=0x7f6174080790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.663+0000 7f617ae00700 1 -- 172.21.15.106:0/1015504697 shutdown_connections 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.663+0000 7f617ae00700 1 -- 172.21.15.106:0/1015504697 wait complete. 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.664+0000 7f617ae00700 1 Processor -- start 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.664+0000 7f617ae00700 1 -- start start 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.664+0000 7f617ae00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f61741a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.664+0000 7f617ae00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f61741136b0 con 0x7f617410fc50 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.664+0000 7f6173e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f61741a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.664+0000 7f6173e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f61741a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49954/0 (socket says 172.21.15.106:49954) 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.664+0000 7f6173e00700 1 -- 172.21.15.106:0/549850206 learned_addr learned my addr 172.21.15.106:0/549850206 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:53.328 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.665+0000 7f6173e00700 1 -- 172.21.15.106:0/549850206 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61741a37c0 con 0x7f617410fc50 2024-04-28T12:39:53.329 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.665+0000 7f6173e00700 1 --2- 172.21.15.106:0/549850206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f61741a3280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6164009fb0 tx=0x7f6164011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:53.329 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.665+0000 7f6170c00700 1 -- 172.21.15.106:0/549850206 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61640036a0 con 0x7f617410fc50 2024-04-28T12:39:53.329 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.665+0000 7f617ae00700 1 -- 172.21.15.106:0/549850206 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61741a3a50 con 0x7f617410fc50 2024-04-28T12:39:53.329 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.665+0000 7f6170c00700 1 -- 172.21.15.106:0/549850206 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f616401a430 con 0x7f617410fc50 2024-04-28T12:39:53.330 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.665+0000 7f6170c00700 1 -- 172.21.15.106:0/549850206 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6164019550 con 0x7f617410fc50 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.666+0000 7f617ae00700 1 -- 172.21.15.106:0/549850206 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61741a6740 con 0x7f617410fc50 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.666+0000 7f6170c00700 1 -- 172.21.15.106:0/549850206 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6164019b90 con 0x7f617410fc50 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.666+0000 7f6170c00700 1 -- 172.21.15.106:0/549850206 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f61640248a0 con 0x7f617410fc50 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.667+0000 7f617ae00700 1 -- 172.21.15.106:0/549850206 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6174064a80 con 0x7f617410fc50 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.669+0000 7f6170c00700 1 -- 172.21.15.106:0/549850206 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6164043ec0 con 0x7f617410fc50 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.746+0000 7f617ae00700 1 -- 172.21.15.106:0/549850206 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f61741a6c90 con 0x7f617410fc50 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.747+0000 7f6170c00700 1 -- 172.21.15.106:0/549850206 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f6164032020 con 0x7f617410fc50 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.748+0000 7f617ae00700 1 -- 172.21.15.106:0/549850206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 msgr2=0x7f61741a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.748+0000 7f617ae00700 1 --2- 172.21.15.106:0/549850206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f61741a3280 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6164009fb0 tx=0x7f6164011940 comp rx=0 tx=0).stop 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.748+0000 7f617ae00700 1 -- 172.21.15.106:0/549850206 shutdown_connections 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.748+0000 7f617ae00700 1 --2- 172.21.15.106:0/549850206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f617410fc50 0x7f61741a3280 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.748+0000 7f617ae00700 1 -- 172.21.15.106:0/549850206 >> 172.21.15.106:0/549850206 conn(0x7f617407e370 msgr2=0x7f617407eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.748+0000 7f617ae00700 1 -- 172.21.15.106:0/549850206 shutdown_connections 2024-04-28T12:39:53.331 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:52.748+0000 7f617ae00700 1 -- 172.21.15.106:0/549850206 wait complete. 2024-04-28T12:39:53.332 INFO:teuthology.orchestra.run.smithi106.stdout:Restarting the monitor... 2024-04-28T12:39:53.815 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:53 smithi106 systemd[1]: Stopping Ceph mon.a for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:39:54.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a[27709]: 2024-04-28T12:39:53.817+0000 7f37cc000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T12:39:54.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a[27709]: 2024-04-28T12:39:53.817+0000 7f37cc000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-28T12:39:54.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:53 smithi106 podman[27957]: 2024-04-28 12:39:53.972482414 +0000 UTC m=+0.272658282 container died 4af5eddd69c55db75e5b37aa8a462ce18e13b19932fe06cd9d3362b298333c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-28T12:39:54.557 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:54 smithi106 podman[27957]: 2024-04-28 12:39:54.385330901 +0000 UTC m=+0.685506780 container cleanup 4af5eddd69c55db75e5b37aa8a462ce18e13b19932fe06cd9d3362b298333c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T12:39:54.558 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:54 smithi106 bash[27957]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a 2024-04-28T12:39:54.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:54 smithi106 podman[27971]: 2024-04-28 12:39:54.561125399 +0000 UTC m=+0.573526685 container remove 4af5eddd69c55db75e5b37aa8a462ce18e13b19932fe06cd9d3362b298333c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-28T12:39:55.300 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 systemd[1]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-28T12:39:55.300 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 systemd[1]: Stopped Ceph mon.a for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:39:55.300 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 systemd[1]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.881s CPU time. 2024-04-28T12:39:55.300 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 systemd[1]: Starting Ceph mon.a for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:39:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 podman[28063]: 2024-04-28 12:39:55.351922413 +0000 UTC m=+0.032336773 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:39:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 podman[28063]: 2024-04-28 12:39:55.446466439 +0000 UTC m=+0.126880820 container create d6af359f5189fcbd9d4732bbc68c339405cfa8ca36afb66efb3c3f83ece8ccfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T12:39:55.764 INFO:teuthology.orchestra.run.smithi106.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-28T12:39:56.042 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 podman[28063]: 2024-04-28 12:39:55.69166606 +0000 UTC m=+0.372080481 container init d6af359f5189fcbd9d4732bbc68c339405cfa8ca36afb66efb3c3f83ece8ccfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-28T12:39:56.042 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 podman[28063]: 2024-04-28 12:39:55.701672538 +0000 UTC m=+0.382086926 container start d6af359f5189fcbd9d4732bbc68c339405cfa8ca36afb66efb3c3f83ece8ccfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: set uid:gid to 167:167 (ceph:ceph) 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev), process ceph-mon, pid 2 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: pidfile_write: ignore empty --pid-file 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: load: jerasure load: lrc 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 bash[28063]: d6af359f5189fcbd9d4732bbc68c339405cfa8ca36afb66efb3c3f83ece8ccfc 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: RocksDB version: 7.9.2 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Git sha 0 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Compile date 2024-04-26 11:39:23 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: DB SUMMARY 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: DB Session ID: 0JKHU6V7ITONXL6YPMMB 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: CURRENT file: CURRENT 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: IDENTITY file: IDENTITY 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-28T12:39:56.043 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104142 ; 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.error_if_exists: 0 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.create_if_missing: 0 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.paranoid_checks: 1 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.env: 0x564021330cc0 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.fs: PosixFileSystem 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.info_log: 0x56404604ddc0 2024-04-28T12:39:56.044 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_file_opening_threads: 16 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.statistics: (nil) 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.use_fsync: 0 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_log_file_size: 0 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.keep_log_file_num: 1000 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.recycle_log_file_num: 0 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.allow_fallocate: 1 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.allow_mmap_reads: 0 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.allow_mmap_writes: 0 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.use_direct_reads: 0 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.create_missing_column_families: 0 2024-04-28T12:39:56.045 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.db_log_dir: 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.wal_dir: 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.advise_random_on_open: 1 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.db_write_buffer_size: 0 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.write_buffer_manager: 0x56404601f0e0 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-28T12:39:56.046 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.rate_limiter: (nil) 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.wal_recovery_mode: 2 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.enable_thread_tracking: 0 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.enable_pipelined_write: 0 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.unordered_write: 0 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-28T12:39:56.047 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.row_cache: None 2024-04-28T12:39:56.048 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.wal_filter: None 2024-04-28T12:39:56.048 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-28T12:39:56.048 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.allow_ingest_behind: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.two_write_queues: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.manual_wal_flush: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.wal_compression: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.atomic_flush: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.log_readahead_size: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.best_efforts_recovery: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.allow_data_in_errors: 0 2024-04-28T12:39:56.049 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.db_host_id: __hostname__ 2024-04-28T12:39:56.050 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-28T12:39:56.050 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_background_jobs: 2 2024-04-28T12:39:56.050 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_background_compactions: -1 2024-04-28T12:39:56.050 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_subcompactions: 1 2024-04-28T12:39:56.050 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-28T12:39:56.050 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-28T12:39:56.050 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-28T12:39:56.050 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_total_wal_size: 0 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_open_files: -1 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bytes_per_sync: 0 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_readahead_size: 0 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_background_flushes: -1 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Compression algorithms supported: 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: kZSTD supported: 0 2024-04-28T12:39:56.051 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: kXpressCompression supported: 0 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: kLZ4HCCompression supported: 1 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: kZlibCompression supported: 1 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: kSnappyCompression supported: 1 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: kLZ4Compression supported: 1 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: kBZip2Compression supported: 0 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.merge_operator: 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_filter: None 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_filter_factory: None 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.sst_partitioner_factory: None 2024-04-28T12:39:56.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56404604df80) 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: cache_index_and_filter_blocks: 1 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: pin_top_level_index_and_filter: 1 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: index_type: 0 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: data_block_index_type: 0 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: index_shortening: 1 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: checksum: 4 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: no_block_cache: 0 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: block_cache: 0x5640460511f0 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: block_cache_name: BinnedLRUCache 2024-04-28T12:39:56.053 INFO:journalctl@ceph.mon.a.smithi106.stdout: block_cache_options: 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: capacity : 536870912 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: num_shard_bits : 4 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: strict_capacity_limit : 0 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: high_pri_pool_ratio: 0.000 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: block_cache_compressed: (nil) 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: persistent_cache: (nil) 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: block_size: 4096 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: block_size_deviation: 10 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: block_restart_interval: 16 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: index_block_restart_interval: 1 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: metadata_block_size: 4096 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: partition_filters: 0 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: use_delta_encoding: 1 2024-04-28T12:39:56.054 INFO:journalctl@ceph.mon.a.smithi106.stdout: filter_policy: bloomfilter 2024-04-28T12:39:56.055 INFO:journalctl@ceph.mon.a.smithi106.stdout: whole_key_filtering: 1 2024-04-28T12:39:56.055 INFO:journalctl@ceph.mon.a.smithi106.stdout: verify_compression: 0 2024-04-28T12:39:56.055 INFO:journalctl@ceph.mon.a.smithi106.stdout: read_amp_bytes_per_bit: 0 2024-04-28T12:39:56.055 INFO:journalctl@ceph.mon.a.smithi106.stdout: format_version: 5 2024-04-28T12:39:56.055 INFO:journalctl@ceph.mon.a.smithi106.stdout: enable_index_compression: 1 2024-04-28T12:39:56.055 INFO:journalctl@ceph.mon.a.smithi106.stdout: block_align: 0 2024-04-28T12:39:56.055 INFO:journalctl@ceph.mon.a.smithi106.stdout: max_auto_readahead_size: 262144 2024-04-28T12:39:56.055 INFO:journalctl@ceph.mon.a.smithi106.stdout: prepopulate_block_cache: 0 2024-04-28T12:39:56.055 INFO:journalctl@ceph.mon.a.smithi106.stdout: initial_auto_readahead_size: 8192 2024-04-28T12:39:56.056 INFO:journalctl@ceph.mon.a.smithi106.stdout: num_file_reads_for_auto_readahead: 2 2024-04-28T12:39:56.056 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.write_buffer_size: 33554432 2024-04-28T12:39:56.056 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_write_buffer_number: 2 2024-04-28T12:39:56.056 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression: NoCompression 2024-04-28T12:39:56.056 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression: Disabled 2024-04-28T12:39:56.056 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.prefix_extractor: nullptr 2024-04-28T12:39:56.056 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-28T12:39:56.056 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.num_levels: 7 2024-04-28T12:39:56.056 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression_opts.level: 32767 2024-04-28T12:39:56.057 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression_opts.strategy: 0 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression_opts.enabled: false 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.target_file_size_base: 67108864 2024-04-28T12:39:56.058 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-28T12:39:56.059 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.arena_block_size: 1048576 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.disable_auto_compactions: 0 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-28T12:39:56.060 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.inplace_update_support: 0 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.bloom_locality: 0 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.max_successive_merges: 0 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.paranoid_file_checks: 0 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.force_consistency_checks: 1 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.report_bg_io_stats: 0 2024-04-28T12:39:56.061 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.ttl: 2592000 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.enable_blob_files: false 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.min_blob_size: 0 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.blob_file_size: 268435456 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.blob_file_starting_level: 0 2024-04-28T12:39:56.062 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c7e94972-9f75-478d-90ca-32c7ead70cc3 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714307995761713, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714307995763430, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99377, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97323, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13927, "raw_average_key_size": 49, "raw_value_size": 89700, "raw_average_value_size": 319, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714307995, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c7e94972-9f75-478d-90ca-32c7ead70cc3", "db_session_id": "0JKHU6V7ITONXL6YPMMB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714307995763617, "job": 1, "event": "recovery_finished"} 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 systemd[1]: Started Ceph mon.a for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56404616e000 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: DB pointer 0x56404615a000 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-28T12:39:56.063 INFO:journalctl@ceph.mon.a.smithi106.stdout: ** DB Stats ** 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: ** Compaction Stats [default] ** 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-28T12:39:56.064 INFO:journalctl@ceph.mon.a.smithi106.stdout: L0 2/0 98.91 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: Sum 2/0 98.91 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 82.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: ** Compaction Stats [default] ** 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 82.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-28T12:39:56.065 INFO:journalctl@ceph.mon.a.smithi106.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout: Cumulative compaction: 0.00 GB write, 12.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout: Interval compaction: 0.00 GB write, 12.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout: Block cache BinnedLRUCache@0x5640460511f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.3e-05 secs_since: 0 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout: 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-28T12:39:56.066 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: starting mon.a rank 0 at public addrs [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] at bind addrs [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mon.a@-1(???) e1 preinit fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mon.a@-1(???).mds e1 new map 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mon.a@-1(???).mds e1 print_map 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout: e1 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout: legacy client fscid: -1 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout: 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout: No filesystems configured 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-28T12:39:56.067 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: monmap epoch 1 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: last_changed 2024-04-28T12:39:45.237228+0000 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: min_mon_release 19 (squid) 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: election_strategy: 1 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: fsmap 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: osdmap e1: 0 total, 0 up, 0 in 2024-04-28T12:39:56.068 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:55 smithi106 ceph-mon[28081]: mgrmap e1: no daemons active 2024-04-28T12:39:56.999 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.378+0000 7f545c800700 1 Processor -- start 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.378+0000 7f545c800700 1 -- start start 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.378+0000 7f545c800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545407d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.379+0000 7f545c800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f545407d830 con 0x7f545407ee00 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.379+0000 7f5459e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545407d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.379+0000 7f5459e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545407d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49960/0 (socket says 172.21.15.106:49960) 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.379+0000 7f5459e00700 1 -- 172.21.15.106:0/4163674222 learned_addr learned my addr 172.21.15.106:0/4163674222 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.380+0000 7f5459e00700 1 -- 172.21.15.106:0/4163674222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545407d9b0 con 0x7f545407ee00 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.380+0000 7f5459e00700 1 --2- 172.21.15.106:0/4163674222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545407d260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f54480098a0 tx=0x7f5448009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79fca3a98afd213 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.380+0000 7f5458a00700 1 -- 172.21.15.106:0/4163674222 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5448004160 con 0x7f545407ee00 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.381+0000 7f5458a00700 1 -- 172.21.15.106:0/4163674222 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5448004300 con 0x7f545407ee00 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.381+0000 7f5458a00700 1 -- 172.21.15.106:0/4163674222 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5448011af0 con 0x7f545407ee00 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.381+0000 7f545c800700 1 -- 172.21.15.106:0/4163674222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 msgr2=0x7f545407d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:57.000 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.381+0000 7f545c800700 1 --2- 172.21.15.106:0/4163674222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545407d260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f54480098a0 tx=0x7f5448009bb0 comp rx=0 tx=0).stop 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.381+0000 7f545c800700 1 -- 172.21.15.106:0/4163674222 shutdown_connections 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.381+0000 7f545c800700 1 --2- 172.21.15.106:0/4163674222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545407d260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.381+0000 7f545c800700 1 -- 172.21.15.106:0/4163674222 >> 172.21.15.106:0/4163674222 conn(0x7f54541042c0 msgr2=0x7f54541066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.382+0000 7f545c800700 1 -- 172.21.15.106:0/4163674222 shutdown_connections 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.382+0000 7f545c800700 1 -- 172.21.15.106:0/4163674222 wait complete. 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.383+0000 7f545c800700 1 Processor -- start 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.383+0000 7f545c800700 1 -- start start 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.383+0000 7f545c800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.383+0000 7f545c800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f545410f2d0 con 0x7f545407ee00 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.383+0000 7f5459e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.383+0000 7f5459e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545419ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49970/0 (socket says 172.21.15.106:49970) 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.383+0000 7f5459e00700 1 -- 172.21.15.106:0/2114459072 learned_addr learned my addr 172.21.15.106:0/2114459072 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:57.001 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.384+0000 7f5459e00700 1 -- 172.21.15.106:0/2114459072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545419f3b0 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.384+0000 7f5459e00700 1 --2- 172.21.15.106:0/2114459072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545419ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5448011ac0 tx=0x7f5448011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.384+0000 7f544ea00700 1 -- 172.21.15.106:0/2114459072 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5448003970 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.384+0000 7f545c800700 1 -- 172.21.15.106:0/2114459072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545419f640 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.385+0000 7f545c800700 1 -- 172.21.15.106:0/2114459072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54541a2330 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.385+0000 7f544ea00700 1 -- 172.21.15.106:0/2114459072 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5448003b10 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.385+0000 7f544ea00700 1 -- 172.21.15.106:0/2114459072 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5448024610 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.385+0000 7f544ea00700 1 -- 172.21.15.106:0/2114459072 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5448024c20 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.385+0000 7f544ea00700 1 -- 172.21.15.106:0/2114459072 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5448019700 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.386+0000 7f545c800700 1 -- 172.21.15.106:0/2114459072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5454064a80 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.389+0000 7f544ea00700 1 -- 172.21.15.106:0/2114459072 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f544801a9e0 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.464+0000 7f545c800700 1 -- 172.21.15.106:0/2114459072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f54541a2500 con 0x7f545407ee00 2024-04-28T12:39:57.002 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.467+0000 7f544ea00700 1 -- 172.21.15.106:0/2114459072 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f544801a430 con 0x7f545407ee00 2024-04-28T12:39:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.468+0000 7f545c800700 1 -- 172.21.15.106:0/2114459072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 msgr2=0x7f545419ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.468+0000 7f545c800700 1 --2- 172.21.15.106:0/2114459072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545419ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5448011ac0 tx=0x7f5448011ba0 comp rx=0 tx=0).stop 2024-04-28T12:39:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.468+0000 7f545c800700 1 -- 172.21.15.106:0/2114459072 shutdown_connections 2024-04-28T12:39:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.468+0000 7f545c800700 1 --2- 172.21.15.106:0/2114459072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f545407ee00 0x7f545419ee70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.468+0000 7f545c800700 1 -- 172.21.15.106:0/2114459072 >> 172.21.15.106:0/2114459072 conn(0x7f54541042c0 msgr2=0x7f5454104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.468+0000 7f545c800700 1 -- 172.21.15.106:0/2114459072 shutdown_connections 2024-04-28T12:39:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:56.468+0000 7f545c800700 1 -- 172.21.15.106:0/2114459072 wait complete. 2024-04-28T12:39:57.003 INFO:teuthology.orchestra.run.smithi106.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-28T12:39:57.046 INFO:teuthology.orchestra.run.smithi106.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-28T12:39:57.046 INFO:teuthology.orchestra.run.smithi106.stdout:Creating mgr... 2024-04-28T12:39:57.046 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-28T12:39:57.047 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-28T12:39:57.480 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.x 2024-04-28T12:39:57.480 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-28T12:39:57.737 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968.target.wants/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@.service. 2024-04-28T12:39:57.779 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:57 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2114459072' entity='client.admin' 2024-04-28T12:39:58.078 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:57 smithi106 systemd[1]: Starting Ceph mgr.x for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:39:58.385 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:58 smithi106 podman[28294]: 2024-04-28 12:39:58.039392852 +0000 UTC m=+0.033189775 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-28T12:39:58.385 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:58 smithi106 podman[28294]: 2024-04-28 12:39:58.142259281 +0000 UTC m=+0.136056222 container create 489a4dc1e7dce79d543fd6675215a3930fbcc2a6e4c815e4a6baf58c91de9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-28T12:39:58.386 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:58 smithi106 podman[28294]: 2024-04-28 12:39:58.387524349 +0000 UTC m=+0.381321333 container init 489a4dc1e7dce79d543fd6675215a3930fbcc2a6e4c815e4a6baf58c91de9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-28T12:39:58.509 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-28T12:39:58.509 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout disabled 2024-04-28T12:39:58.527 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-28T12:39:58.527 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout inactive 2024-04-28T12:39:58.527 INFO:teuthology.orchestra.run.smithi106.stdout:firewalld.service is not enabled 2024-04-28T12:39:58.527 INFO:teuthology.orchestra.run.smithi106.stdout:Not possible to enable service . firewalld.service is not available 2024-04-28T12:39:58.549 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-28T12:39:58.549 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout disabled 2024-04-28T12:39:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-28T12:39:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout inactive 2024-04-28T12:39:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:firewalld.service is not enabled 2024-04-28T12:39:58.567 INFO:teuthology.orchestra.run.smithi106.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-28T12:39:58.568 INFO:teuthology.orchestra.run.smithi106.stdout:Waiting for mgr to start... 2024-04-28T12:39:58.568 INFO:teuthology.orchestra.run.smithi106.stdout:Waiting for mgr... 2024-04-28T12:39:58.645 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:58 smithi106 podman[28294]: 2024-04-28 12:39:58.399236015 +0000 UTC m=+0.393032970 container start 489a4dc1e7dce79d543fd6675215a3930fbcc2a6e4c815e4a6baf58c91de9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-28T12:39:58.645 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:58 smithi106 bash[28294]: 489a4dc1e7dce79d543fd6675215a3930fbcc2a6e4c815e4a6baf58c91de9ea6 2024-04-28T12:39:58.646 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:58 smithi106 systemd[1]: Started Ceph mgr.x for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:39:58.905 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:39:58.648+0000 7fdfb4450200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-28T12:39:58.905 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:39:58.907+0000 7fdfb4450200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:39:59.169 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:59 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:39:59.039+0000 7fdfb4450200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-28T12:39:59.449 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:39:59 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2703311662' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-28T12:39:59.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:59 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:39:59.452+0000 7fdfb4450200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-28T12:39:59.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:39:59 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:39:59.569+0000 7fdfb4450200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-28T12:39:59.929 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:39:59.931 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout { 2024-04-28T12:39:59.931 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:39:59.931 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "health": { 2024-04-28T12:39:59.931 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-28T12:39:59.931 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-28T12:39:59.932 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-28T12:39:59.935 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 0 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "a" 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:39:59.936 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-28T12:39:59.937 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-28T12:39:59.938 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-28T12:39:59.938 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-28T12:39:59.938 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:39:59.938 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-28T12:39:59.938 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:39:59.938 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-28T12:39:59.938 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-28T12:39:59.944 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:39:59.944 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-28T12:39:59.944 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "restful" 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "modified": "2024-04-28T12:39:49.394447+0000", 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-28T12:39:59.945 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout } 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.171+0000 7ff938e00700 1 Processor -- start 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.172+0000 7ff938e00700 1 -- start start 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.172+0000 7ff938e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff93410bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.172+0000 7ff938e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff93407cdd0 con 0x7ff9341097f0 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.173+0000 7ff931e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff93410bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.173+0000 7ff931e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff93410bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41700/0 (socket says 172.21.15.106:41700) 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.173+0000 7ff931e00700 1 -- 172.21.15.106:0/1372477945 learned_addr learned my addr 172.21.15.106:0/1372477945 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.173+0000 7ff931e00700 1 -- 172.21.15.106:0/1372477945 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93410c120 con 0x7ff9341097f0 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.174+0000 7ff931e00700 1 --2- 172.21.15.106:0/1372477945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff93410bbe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff9240098a0 tx=0x7ff924009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82c44c483ffcc1e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.174+0000 7ff930a00700 1 -- 172.21.15.106:0/1372477945 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff924004160 con 0x7ff9341097f0 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.174+0000 7ff930a00700 1 -- 172.21.15.106:0/1372477945 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff924004300 con 0x7ff9341097f0 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.174+0000 7ff930a00700 1 -- 172.21.15.106:0/1372477945 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff924011af0 con 0x7ff9341097f0 2024-04-28T12:39:59.946 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.175+0000 7ff938e00700 1 -- 172.21.15.106:0/1372477945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 msgr2=0x7ff93410bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.175+0000 7ff938e00700 1 --2- 172.21.15.106:0/1372477945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff93410bbe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff9240098a0 tx=0x7ff924009bb0 comp rx=0 tx=0).stop 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.175+0000 7ff938e00700 1 -- 172.21.15.106:0/1372477945 shutdown_connections 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.175+0000 7ff938e00700 1 --2- 172.21.15.106:0/1372477945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff93410bbe0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.175+0000 7ff938e00700 1 -- 172.21.15.106:0/1372477945 >> 172.21.15.106:0/1372477945 conn(0x7ff9341031e0 msgr2=0x7ff934105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.175+0000 7ff938e00700 1 -- 172.21.15.106:0/1372477945 shutdown_connections 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.175+0000 7ff938e00700 1 -- 172.21.15.106:0/1372477945 wait complete. 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.176+0000 7ff938e00700 1 Processor -- start 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.176+0000 7ff938e00700 1 -- start start 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.176+0000 7ff938e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff934109070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.176+0000 7ff938e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff93410d090 con 0x7ff9341097f0 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.177+0000 7ff931e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff934109070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.177+0000 7ff931e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff934109070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41704/0 (socket says 172.21.15.106:41704) 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.177+0000 7ff931e00700 1 -- 172.21.15.106:0/2703311662 learned_addr learned my addr 172.21.15.106:0/2703311662 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:39:59.947 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.177+0000 7ff931e00700 1 -- 172.21.15.106:0/2703311662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9341095b0 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.177+0000 7ff931e00700 1 --2- 172.21.15.106:0/2703311662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff934109070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff924011ac0 tx=0x7ff924011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.178+0000 7ff92aa00700 1 -- 172.21.15.106:0/2703311662 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff924003970 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.178+0000 7ff92aa00700 1 -- 172.21.15.106:0/2703311662 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff924003b10 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.178+0000 7ff92aa00700 1 -- 172.21.15.106:0/2703311662 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff924024610 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.178+0000 7ff938e00700 1 -- 172.21.15.106:0/2703311662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9341077a0 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.178+0000 7ff938e00700 1 -- 172.21.15.106:0/2703311662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff934107ce0 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.178+0000 7ff92aa00700 1 -- 172.21.15.106:0/2703311662 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff924024c20 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.178+0000 7ff92aa00700 1 -- 172.21.15.106:0/2703311662 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff924019700 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.179+0000 7ff938e00700 1 -- 172.21.15.106:0/2703311662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff934064a80 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.181+0000 7ff92aa00700 1 -- 172.21.15.106:0/2703311662 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff92401a9e0 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.261+0000 7ff938e00700 1 -- 172.21.15.106:0/2703311662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff934108020 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.262+0000 7ff92aa00700 1 -- 172.21.15.106:0/2703311662 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff92401a430 con 0x7ff9341097f0 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.264+0000 7ff938e00700 1 -- 172.21.15.106:0/2703311662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 msgr2=0x7ff934109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:39:59.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.264+0000 7ff938e00700 1 --2- 172.21.15.106:0/2703311662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff934109070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff924011ac0 tx=0x7ff924011ba0 comp rx=0 tx=0).stop 2024-04-28T12:39:59.949 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.264+0000 7ff938e00700 1 -- 172.21.15.106:0/2703311662 shutdown_connections 2024-04-28T12:39:59.949 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.264+0000 7ff938e00700 1 --2- 172.21.15.106:0/2703311662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9341097f0 0x7ff934109070 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:39:59.949 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.264+0000 7ff938e00700 1 -- 172.21.15.106:0/2703311662 >> 172.21.15.106:0/2703311662 conn(0x7ff9341031e0 msgr2=0x7ff934103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:39:59.949 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.264+0000 7ff938e00700 1 -- 172.21.15.106:0/2703311662 shutdown_connections 2024-04-28T12:39:59.949 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:39:59.264+0000 7ff938e00700 1 -- 172.21.15.106:0/2703311662 wait complete. 2024-04-28T12:39:59.949 INFO:teuthology.orchestra.run.smithi106.stdout:mgr not available, waiting (1/15)... 2024-04-28T12:40:01.573 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:01.133+0000 7fdfb4450200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-28T12:40:01.573 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:01.355+0000 7fdfb4450200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-28T12:40:01.573 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:01.575+0000 7fdfb4450200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-28T12:40:02.202 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:01.949+0000 7fdfb4450200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-28T12:40:02.202 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:02.075+0000 7fdfb4450200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-28T12:40:02.498 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:02.347+0000 7fdfb4450200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-28T12:40:02.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:02 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3233035435' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-28T12:40:02.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:02.608+0000 7fdfb4450200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:40:03.236 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:40:03.237 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout { 2024-04-28T12:40:03.237 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:40:03.237 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "health": { 2024-04-28T12:40:03.237 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-28T12:40:03.238 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-28T12:40:03.238 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-28T12:40:03.239 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:03.239 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-28T12:40:03.239 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-28T12:40:03.239 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 0 2024-04-28T12:40:03.239 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:40:03.239 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-28T12:40:03.240 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "a" 2024-04-28T12:40:03.240 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-28T12:40:03.241 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-28T12:40:03.249 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-28T12:40:03.249 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-28T12:40:03.249 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-28T12:40:03.249 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-28T12:40:03.250 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "restful" 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:03.251 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "modified": "2024-04-28T12:39:49.394447+0000", 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout } 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.501+0000 7f9255c00700 1 Processor -- start 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.501+0000 7f9255c00700 1 -- start start 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.501+0000 7f9255c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925007d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.501+0000 7f9255c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f925007d850 con 0x7f925007ee20 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.502+0000 7f924ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925007d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.502+0000 7f924ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925007d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41720/0 (socket says 172.21.15.106:41720) 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.502+0000 7f924ea00700 1 -- 172.21.15.106:0/1729649871 learned_addr learned my addr 172.21.15.106:0/1729649871 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.502+0000 7f924ea00700 1 -- 172.21.15.106:0/1729649871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925007d9d0 con 0x7f925007ee20 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.503+0000 7f924ea00700 1 --2- 172.21.15.106:0/1729649871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925007d280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92400098a0 tx=0x7f9240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb836f2055f0f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:03.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.503+0000 7f924d600700 1 -- 172.21.15.106:0/1729649871 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9240004160 con 0x7f925007ee20 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.503+0000 7f924d600700 1 -- 172.21.15.106:0/1729649871 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9240004300 con 0x7f925007ee20 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.503+0000 7f924d600700 1 -- 172.21.15.106:0/1729649871 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9240011af0 con 0x7f925007ee20 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.504+0000 7f9255c00700 1 -- 172.21.15.106:0/1729649871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 msgr2=0x7f925007d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.504+0000 7f9255c00700 1 --2- 172.21.15.106:0/1729649871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925007d280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f92400098a0 tx=0x7f9240009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.504+0000 7f9255c00700 1 -- 172.21.15.106:0/1729649871 shutdown_connections 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.504+0000 7f9255c00700 1 --2- 172.21.15.106:0/1729649871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925007d280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.504+0000 7f9255c00700 1 -- 172.21.15.106:0/1729649871 >> 172.21.15.106:0/1729649871 conn(0x7f92501042f0 msgr2=0x7f9250106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.504+0000 7f9255c00700 1 -- 172.21.15.106:0/1729649871 shutdown_connections 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.504+0000 7f9255c00700 1 -- 172.21.15.106:0/1729649871 wait complete. 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.505+0000 7f9255c00700 1 Processor -- start 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.506+0000 7f9255c00700 1 -- start start 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.506+0000 7f9255c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.506+0000 7f9255c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f925010f2c0 con 0x7f925007ee20 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.506+0000 7f924ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.506+0000 7f924ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41728/0 (socket says 172.21.15.106:41728) 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.506+0000 7f924ea00700 1 -- 172.21.15.106:0/3233035435 learned_addr learned my addr 172.21.15.106:0/3233035435 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.507+0000 7f924ea00700 1 -- 172.21.15.106:0/3233035435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925019f420 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.507+0000 7f924ea00700 1 --2- 172.21.15.106:0/3233035435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925019eee0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f92400117b0 tx=0x7f9240011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.507+0000 7f9247400700 1 -- 172.21.15.106:0/3233035435 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9240003af0 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.507+0000 7f9247400700 1 -- 172.21.15.106:0/3233035435 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9240020070 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.507+0000 7f9247400700 1 -- 172.21.15.106:0/3233035435 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9240019630 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.507+0000 7f9255c00700 1 -- 172.21.15.106:0/3233035435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925019f6b0 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.507+0000 7f9255c00700 1 -- 172.21.15.106:0/3233035435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92501a23a0 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.508+0000 7f9247400700 1 -- 172.21.15.106:0/3233035435 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9240019c30 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.508+0000 7f9247400700 1 -- 172.21.15.106:0/3233035435 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f924000b720 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.509+0000 7f9255c00700 1 -- 172.21.15.106:0/3233035435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9250064a80 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.511+0000 7f9247400700 1 -- 172.21.15.106:0/3233035435 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f924001a9c0 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.590+0000 7f9255c00700 1 -- 172.21.15.106:0/3233035435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f92501a28f0 con 0x7f925007ee20 2024-04-28T12:40:03.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.591+0000 7f9247400700 1 -- 172.21.15.106:0/3233035435 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f924001a430 con 0x7f925007ee20 2024-04-28T12:40:03.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.592+0000 7f9255c00700 1 -- 172.21.15.106:0/3233035435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 msgr2=0x7f925019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:03.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.592+0000 7f9255c00700 1 --2- 172.21.15.106:0/3233035435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925019eee0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f92400117b0 tx=0x7f9240011b70 comp rx=0 tx=0).stop 2024-04-28T12:40:03.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.592+0000 7f9255c00700 1 -- 172.21.15.106:0/3233035435 shutdown_connections 2024-04-28T12:40:03.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.592+0000 7f9255c00700 1 --2- 172.21.15.106:0/3233035435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f925007ee20 0x7f925019eee0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:03.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.592+0000 7f9255c00700 1 -- 172.21.15.106:0/3233035435 >> 172.21.15.106:0/3233035435 conn(0x7f92501042f0 msgr2=0x7f9250104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:03.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.593+0000 7f9255c00700 1 -- 172.21.15.106:0/3233035435 shutdown_connections 2024-04-28T12:40:03.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:02.593+0000 7f9255c00700 1 -- 172.21.15.106:0/3233035435 wait complete. 2024-04-28T12:40:03.255 INFO:teuthology.orchestra.run.smithi106.stdout:mgr not available, waiting (2/15)... 2024-04-28T12:40:03.574 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:03 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:03.406+0000 7fdfb4450200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-28T12:40:04.325 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:03 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:03.949+0000 7fdfb4450200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-28T12:40:04.325 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:04 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:04.075+0000 7fdfb4450200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-28T12:40:04.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:04 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:04.338+0000 7fdfb4450200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-28T12:40:05.324 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:04 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:04.975+0000 7fdfb4450200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-28T12:40:05.324 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:05 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:05.095+0000 7fdfb4450200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-28T12:40:05.682 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:05 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:05.415+0000 7fdfb4450200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-28T12:40:05.682 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:05 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:05.595+0000 7fdfb4450200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-28T12:40:05.944 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:05 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:05.723+0000 7fdfb4450200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-28T12:40:05.944 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:05 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:05.880+0000 7fdfb4450200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-28T12:40:06.253 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:05 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4207314655' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-28T12:40:06.574 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:06 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:06.256+0000 7fdfb4450200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout { 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "health": { 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-28T12:40:06.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 0 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "a" 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-28T12:40:06.671 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-28T12:40:06.682 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-28T12:40:06.683 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "restful" 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "modified": "2024-04-28T12:39:49.394447+0000", 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-28T12:40:06.684 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout } 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.836+0000 7fc7a1200700 1 Processor -- start 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.837+0000 7fc7a1200700 1 -- start start 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.837+0000 7fc7a1200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c10bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.837+0000 7fc7a1200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c07cdb0 con 0x7fc79c109810 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.837+0000 7fc799e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c10bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.838+0000 7fc799e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c10bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41730/0 (socket says 172.21.15.106:41730) 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.838+0000 7fc799e00700 1 -- 172.21.15.106:0/3951148027 learned_addr learned my addr 172.21.15.106:0/3951148027 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.838+0000 7fc799e00700 1 -- 172.21.15.106:0/3951148027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c10c140 con 0x7fc79c109810 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.838+0000 7fc799e00700 1 --2- 172.21.15.106:0/3951148027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c10bc00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098a0 tx=0x7fc78c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56d7152d2a3b6d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.839+0000 7fc798a00700 1 -- 172.21.15.106:0/3951148027 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc78c004160 con 0x7fc79c109810 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.839+0000 7fc798a00700 1 -- 172.21.15.106:0/3951148027 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc78c004300 con 0x7fc79c109810 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.839+0000 7fc798a00700 1 -- 172.21.15.106:0/3951148027 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc78c011af0 con 0x7fc79c109810 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.839+0000 7fc7a1200700 1 -- 172.21.15.106:0/3951148027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 msgr2=0x7fc79c10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:06.685 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.839+0000 7fc7a1200700 1 --2- 172.21.15.106:0/3951148027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c10bc00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0098a0 tx=0x7fc78c009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.839+0000 7fc7a1200700 1 -- 172.21.15.106:0/3951148027 shutdown_connections 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.839+0000 7fc7a1200700 1 --2- 172.21.15.106:0/3951148027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c10bc00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.839+0000 7fc7a1200700 1 -- 172.21.15.106:0/3951148027 >> 172.21.15.106:0/3951148027 conn(0x7fc79c1031e0 msgr2=0x7fc79c105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.840+0000 7fc7a1200700 1 -- 172.21.15.106:0/3951148027 shutdown_connections 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.840+0000 7fc7a1200700 1 -- 172.21.15.106:0/3951148027 wait complete. 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.841+0000 7fc7a1200700 1 Processor -- start 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.841+0000 7fc7a1200700 1 -- start start 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.841+0000 7fc7a1200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.841+0000 7fc7a1200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c10d0a0 con 0x7fc79c109810 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.841+0000 7fc799e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.841+0000 7fc799e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41740/0 (socket says 172.21.15.106:41740) 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.841+0000 7fc799e00700 1 -- 172.21.15.106:0/4207314655 learned_addr learned my addr 172.21.15.106:0/4207314655 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.842+0000 7fc799e00700 1 -- 172.21.15.106:0/4207314655 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c19d1e0 con 0x7fc79c109810 2024-04-28T12:40:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.842+0000 7fc799e00700 1 --2- 172.21.15.106:0/4207314655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c19cca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc78c011ac0 tx=0x7fc78c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.842+0000 7fc792a00700 1 -- 172.21.15.106:0/4207314655 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc78c003970 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.842+0000 7fc792a00700 1 -- 172.21.15.106:0/4207314655 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc78c003b10 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.842+0000 7fc7a1200700 1 -- 172.21.15.106:0/4207314655 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79c19d470 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.842+0000 7fc792a00700 1 -- 172.21.15.106:0/4207314655 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc78c024610 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.842+0000 7fc7a1200700 1 -- 172.21.15.106:0/4207314655 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79c1a0160 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.843+0000 7fc792a00700 1 -- 172.21.15.106:0/4207314655 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc78c024c20 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.843+0000 7fc792a00700 1 -- 172.21.15.106:0/4207314655 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc78c019700 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.843+0000 7fc7a1200700 1 -- 172.21.15.106:0/4207314655 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79c064a80 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.848+0000 7fc792a00700 1 -- 172.21.15.106:0/4207314655 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc78c01a9e0 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.927+0000 7fc7a1200700 1 -- 172.21.15.106:0/4207314655 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc79c1a0330 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.928+0000 7fc792a00700 1 -- 172.21.15.106:0/4207314655 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fc78c01a430 con 0x7fc79c109810 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.930+0000 7fc7a1200700 1 -- 172.21.15.106:0/4207314655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 msgr2=0x7fc79c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.930+0000 7fc7a1200700 1 --2- 172.21.15.106:0/4207314655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c19cca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc78c011ac0 tx=0x7fc78c011ba0 comp rx=0 tx=0).stop 2024-04-28T12:40:06.687 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.930+0000 7fc7a1200700 1 -- 172.21.15.106:0/4207314655 shutdown_connections 2024-04-28T12:40:06.688 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.930+0000 7fc7a1200700 1 --2- 172.21.15.106:0/4207314655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc79c109810 0x7fc79c19cca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:06.688 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.930+0000 7fc7a1200700 1 -- 172.21.15.106:0/4207314655 >> 172.21.15.106:0/4207314655 conn(0x7fc79c1031e0 msgr2=0x7fc79c103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:06.688 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.930+0000 7fc7a1200700 1 -- 172.21.15.106:0/4207314655 shutdown_connections 2024-04-28T12:40:06.688 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:05.931+0000 7fc7a1200700 1 -- 172.21.15.106:0/4207314655 wait complete. 2024-04-28T12:40:06.688 INFO:teuthology.orchestra.run.smithi106.stdout:mgr not available, waiting (3/15)... 2024-04-28T12:40:07.311 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:07.020+0000 7fdfb4450200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-28T12:40:07.574 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:07.313+0000 7fdfb4450200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-28T12:40:07.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: Activating manager daemon x 2024-04-28T12:40:07.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: mgrmap e2: x(active, starting, since 0.0119521s) 2024-04-28T12:40:07.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-28T12:40:07.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-28T12:40:07.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-28T12:40:07.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:40:07.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-28T12:40:07.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: Manager daemon x is now available 2024-04-28T12:40:07.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-28T12:40:07.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' 2024-04-28T12:40:07.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-28T12:40:07.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:07 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' 2024-04-28T12:40:08.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:08 smithi106 ceph-mon[28081]: from='mgr.14100 172.21.15.106:0/1215619350' entity='mgr.x' 2024-04-28T12:40:08.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:08 smithi106 ceph-mon[28081]: mgrmap e3: x(active, since 1.01795s) 2024-04-28T12:40:10.437 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:40:10.437 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout { 2024-04-28T12:40:10.437 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:40:10.437 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "health": { 2024-04-28T12:40:10.437 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 0 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "a" 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-28T12:40:10.438 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:10.439 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-28T12:40:10.439 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-28T12:40:10.439 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:10.439 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-28T12:40:10.439 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:10.439 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-28T12:40:10.439 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-28T12:40:10.439 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-28T12:40:10.448 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-28T12:40:10.448 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-28T12:40:10.448 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-28T12:40:10.448 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:10.448 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:10.449 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "restful" 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ], 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-28T12:40:10.450 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "modified": "2024-04-28T12:39:49.394447+0000", 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout }, 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout } 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.319+0000 7f7f35000700 1 Processor -- start 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.319+0000 7f7f35000700 1 -- start start 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.320+0000 7f7f35000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3010b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.320+0000 7f7f35000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3010b7f0 con 0x7f7f3010ae90 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.320+0000 7f7f2de00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3010b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.320+0000 7f7f2de00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3010b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37384/0 (socket says 172.21.15.106:37384) 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.320+0000 7f7f2de00700 1 -- 172.21.15.106:0/2231026748 learned_addr learned my addr 172.21.15.106:0/2231026748 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.321+0000 7f7f2de00700 1 -- 172.21.15.106:0/2231026748 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3010c080 con 0x7f7f3010ae90 2024-04-28T12:40:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.321+0000 7f7f2de00700 1 --2- 172.21.15.106:0/2231026748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3010b2b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f200098a0 tx=0x7f7f20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4796688cac58d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.322+0000 7f7f2ca00700 1 -- 172.21.15.106:0/2231026748 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f20004160 con 0x7f7f3010ae90 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.322+0000 7f7f2ca00700 1 -- 172.21.15.106:0/2231026748 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7f20004300 con 0x7f7f3010ae90 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.322+0000 7f7f2ca00700 1 -- 172.21.15.106:0/2231026748 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f20011af0 con 0x7f7f3010ae90 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.322+0000 7f7f35000700 1 -- 172.21.15.106:0/2231026748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 msgr2=0x7f7f3010b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.322+0000 7f7f35000700 1 --2- 172.21.15.106:0/2231026748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3010b2b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f200098a0 tx=0x7f7f20009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.322+0000 7f7f35000700 1 -- 172.21.15.106:0/2231026748 shutdown_connections 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.322+0000 7f7f35000700 1 --2- 172.21.15.106:0/2231026748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3010b2b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.322+0000 7f7f35000700 1 -- 172.21.15.106:0/2231026748 >> 172.21.15.106:0/2231026748 conn(0x7f7f3007e2a0 msgr2=0x7f7f3007e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.323+0000 7f7f35000700 1 -- 172.21.15.106:0/2231026748 shutdown_connections 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.323+0000 7f7f35000700 1 -- 172.21.15.106:0/2231026748 wait complete. 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.324+0000 7f7f35000700 1 Processor -- start 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.324+0000 7f7f35000700 1 -- start start 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.324+0000 7f7f35000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:10.452 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.324+0000 7f7f35000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3010f340 con 0x7f7f3010ae90 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.324+0000 7f7f2de00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.324+0000 7f7f2de00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3019eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37390/0 (socket says 172.21.15.106:37390) 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.324+0000 7f7f2de00700 1 -- 172.21.15.106:0/2560281513 learned_addr learned my addr 172.21.15.106:0/2560281513 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.325+0000 7f7f2de00700 1 -- 172.21.15.106:0/2560281513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3019f400 con 0x7f7f3010ae90 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.325+0000 7f7f2de00700 1 --2- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3019eec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f200117b0 tx=0x7f7f20011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.325+0000 7f7f26a00700 1 -- 172.21.15.106:0/2560281513 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f20003af0 con 0x7f7f3010ae90 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.325+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3019f690 con 0x7f7f3010ae90 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.325+0000 7f7f26a00700 1 -- 172.21.15.106:0/2560281513 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7f20020070 con 0x7f7f3010ae90 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.325+0000 7f7f26a00700 1 -- 172.21.15.106:0/2560281513 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f20019630 con 0x7f7f3010ae90 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.326+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f301a2380 con 0x7f7f3010ae90 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.327+0000 7f7f26a00700 1 -- 172.21.15.106:0/2560281513 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7f7f2001a9c0 con 0x7f7f3010ae90 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.327+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f30064a80 con 0x7f7f3010ae90 2024-04-28T12:40:10.453 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.327+0000 7f7f26a00700 1 --2- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f7f14039490 0x7f7f1403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.327+0000 7f7f26a00700 1 -- 172.21.15.106:0/2560281513 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7f2004d8d0 con 0x7f7f3010ae90 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.327+0000 7f7f2d400700 1 --2- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f7f14039490 0x7f7f1403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.328+0000 7f7f2d400700 1 --2- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f7f14039490 0x7f7f1403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006fd0 tx=0x7f7f1c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.333+0000 7f7f26a00700 1 -- 172.21.15.106:0/2560281513 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7f200042a0 con 0x7f7f3010ae90 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.376+0000 7f7f26a00700 1 -- 172.21.15.106:0/2560281513 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f7f2001a430 con 0x7f7f3010ae90 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.652+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7f301a2550 con 0x7f7f3010ae90 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.653+0000 7f7f26a00700 1 -- 172.21.15.106:0/2560281513 <== mon.0 v2:172.21.15.106:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7f301a2550 con 0x7f7f3010ae90 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.657+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f7f14039490 msgr2=0x7f7f1403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.657+0000 7f7f35000700 1 --2- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f7f14039490 0x7f7f1403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006fd0 tx=0x7f7f1c006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.657+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 msgr2=0x7f7f3019eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.657+0000 7f7f35000700 1 --2- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3019eec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f200117b0 tx=0x7f7f20011b70 comp rx=0 tx=0).stop 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.657+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 shutdown_connections 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.657+0000 7f7f35000700 1 --2- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f7f14039490 0x7f7f1403b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:10.454 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.657+0000 7f7f35000700 1 --2- 172.21.15.106:0/2560281513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f3010ae90 0x7f7f3019eec0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:10.455 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.657+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 >> 172.21.15.106:0/2560281513 conn(0x7f7f3007e2a0 msgr2=0x7f7f301084a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:10.455 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.658+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 shutdown_connections 2024-04-28T12:40:10.455 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:09.658+0000 7f7f35000700 1 -- 172.21.15.106:0/2560281513 wait complete. 2024-04-28T12:40:10.455 INFO:teuthology.orchestra.run.smithi106.stdout:mgr is available 2024-04-28T12:40:10.597 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:10 smithi106 ceph-mon[28081]: mgrmap e4: x(active, since 2s) 2024-04-28T12:40:10.597 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:10 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2560281513' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-28T12:40:11.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:11 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2241587093' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-28T12:40:11.948 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:40:11.949 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout [global] 2024-04-28T12:40:11.949 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout fsid = e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:40:11.949 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-28T12:40:11.949 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout [osd] 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-28T12:40:11.951 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-28T12:40:11.952 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-28T12:40:11.952 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-28T12:40:11.952 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.074+0000 7f54d0400700 1 Processor -- start 2024-04-28T12:40:11.952 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.075+0000 7f54d0400700 1 -- start start 2024-04-28T12:40:11.952 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.075+0000 7f54d0400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c8109170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:11.952 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.075+0000 7f54d0400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8109740 con 0x7f54c8108d50 2024-04-28T12:40:11.952 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.075+0000 7f54cda00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c8109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.075+0000 7f54cda00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c8109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37400/0 (socket says 172.21.15.106:37400) 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.075+0000 7f54cda00700 1 -- 172.21.15.106:0/799038525 learned_addr learned my addr 172.21.15.106:0/799038525 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.076+0000 7f54cda00700 1 -- 172.21.15.106:0/799038525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c8109f50 con 0x7f54c8108d50 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.076+0000 7f54cda00700 1 --2- 172.21.15.106:0/799038525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c8109170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009ab0 tx=0x7f54bc009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c865b2e3f80eeeb server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.077+0000 7f54c7e00700 1 -- 172.21.15.106:0/799038525 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc004160 con 0x7f54c8108d50 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.077+0000 7f54c7e00700 1 -- 172.21.15.106:0/799038525 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54bc00bc70 con 0x7f54c8108d50 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.077+0000 7f54c7e00700 1 -- 172.21.15.106:0/799038525 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc003c30 con 0x7f54c8108d50 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.077+0000 7f54d0400700 1 -- 172.21.15.106:0/799038525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 msgr2=0x7f54c8109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.077+0000 7f54d0400700 1 --2- 172.21.15.106:0/799038525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c8109170 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009ab0 tx=0x7f54bc009dc0 comp rx=0 tx=0).stop 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.077+0000 7f54d0400700 1 -- 172.21.15.106:0/799038525 shutdown_connections 2024-04-28T12:40:11.957 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.077+0000 7f54d0400700 1 --2- 172.21.15.106:0/799038525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c8109170 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.077+0000 7f54d0400700 1 -- 172.21.15.106:0/799038525 >> 172.21.15.106:0/799038525 conn(0x7f54c81042f0 msgr2=0x7f54c8106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.078+0000 7f54d0400700 1 -- 172.21.15.106:0/799038525 shutdown_connections 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.078+0000 7f54d0400700 1 -- 172.21.15.106:0/799038525 wait complete. 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.078+0000 7f54d0400700 1 Processor -- start 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.079+0000 7f54d0400700 1 -- start start 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.079+0000 7f54d0400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.079+0000 7f54d0400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54c810d300 con 0x7f54c8108d50 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.079+0000 7f54cda00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.079+0000 7f54cda00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37410/0 (socket says 172.21.15.106:37410) 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.079+0000 7f54cda00700 1 -- 172.21.15.106:0/2241587093 learned_addr learned my addr 172.21.15.106:0/2241587093 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.080+0000 7f54cda00700 1 -- 172.21.15.106:0/2241587093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c819d220 con 0x7f54c8108d50 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.080+0000 7f54cda00700 1 --2- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c819cce0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00ba10 tx=0x7f54bc00baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:11.958 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.080+0000 7f54c6000700 1 -- 172.21.15.106:0/2241587093 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc004270 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.080+0000 7f54c6000700 1 -- 172.21.15.106:0/2241587093 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f54bc004410 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.080+0000 7f54c6000700 1 -- 172.21.15.106:0/2241587093 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f54bc0116b0 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.080+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c819d4b0 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.080+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c81a0190 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.081+0000 7f54c6000700 1 -- 172.21.15.106:0/2241587093 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f54bc01baf0 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.081+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c8064a80 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.082+0000 7f54c6000700 1 --2- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f54b0039560 0x7f54b003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.082+0000 7f54c6000700 1 -- 172.21.15.106:0/2241587093 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54bc04de00 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.082+0000 7f54cd000700 1 --2- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f54b0039560 0x7f54b003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.083+0000 7f54cd000700 1 --2- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f54b0039560 0x7f54b003ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f54b4006fd0 tx=0x7f54b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.088+0000 7f54c6000700 1 -- 172.21.15.106:0/2241587093 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f54bc01f030 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.303+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f54c81a0360 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.304+0000 7f54c6000700 1 -- 172.21.15.106:0/2241587093 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f54c81a0360 con 0x7f54c8108d50 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f54b0039560 msgr2=0x7f54b003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:11.959 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 --2- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f54b0039560 0x7f54b003ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f54b4006fd0 tx=0x7f54b4006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:11.960 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 msgr2=0x7f54c819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:11.960 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 --2- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c819cce0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00ba10 tx=0x7f54bc00baf0 comp rx=0 tx=0).stop 2024-04-28T12:40:11.960 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 shutdown_connections 2024-04-28T12:40:11.960 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 --2- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f54b0039560 0x7f54b003ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:11.960 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 --2- 172.21.15.106:0/2241587093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c8108d50 0x7f54c819cce0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:11.960 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 >> 172.21.15.106:0/2241587093 conn(0x7f54c81042f0 msgr2=0x7f54c8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:11.960 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 shutdown_connections 2024-04-28T12:40:11.961 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:11.307+0000 7f54d0400700 1 -- 172.21.15.106:0/2241587093 wait complete. 2024-04-28T12:40:11.961 INFO:teuthology.orchestra.run.smithi106.stdout:Enabling cephadm module... 2024-04-28T12:40:13.220 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:12 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3127717486' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-28T12:40:13.220 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: ignoring --setuser ceph since I am not root 2024-04-28T12:40:13.220 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: ignoring --setgroup ceph since I am not root 2024-04-28T12:40:13.575 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:13.223+0000 7fbaf854e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-28T12:40:13.575 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:13.463+0000 7fbaf854e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:40:13.676 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.574+0000 7efe55600700 1 Processor -- start 2024-04-28T12:40:13.676 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.574+0000 7efe55600700 1 -- start start 2024-04-28T12:40:13.676 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.575+0000 7efe55600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe50112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:13.676 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.575+0000 7efe55600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efe50112580 con 0x7efe5010fc50 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.575+0000 7efe4ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe50112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.576+0000 7efe4ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe50112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37420/0 (socket says 172.21.15.106:37420) 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.576+0000 7efe4ea00700 1 -- 172.21.15.106:0/1199703516 learned_addr learned my addr 172.21.15.106:0/1199703516 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.576+0000 7efe4ea00700 1 -- 172.21.15.106:0/1199703516 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe50112700 con 0x7efe5010fc50 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.576+0000 7efe4ea00700 1 --2- 172.21.15.106:0/1199703516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe50112040 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efe400098a0 tx=0x7efe40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c46d8b64d71850 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.577+0000 7efe4e000700 1 -- 172.21.15.106:0/1199703516 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe40004160 con 0x7efe5010fc50 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.577+0000 7efe4e000700 1 -- 172.21.15.106:0/1199703516 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe40004300 con 0x7efe5010fc50 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.577+0000 7efe4e000700 1 -- 172.21.15.106:0/1199703516 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe40011af0 con 0x7efe5010fc50 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.577+0000 7efe55600700 1 -- 172.21.15.106:0/1199703516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 msgr2=0x7efe50112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.577+0000 7efe55600700 1 --2- 172.21.15.106:0/1199703516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe50112040 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efe400098a0 tx=0x7efe40009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.578+0000 7efe55600700 1 -- 172.21.15.106:0/1199703516 shutdown_connections 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.578+0000 7efe55600700 1 --2- 172.21.15.106:0/1199703516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe50112040 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:13.677 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.578+0000 7efe55600700 1 -- 172.21.15.106:0/1199703516 >> 172.21.15.106:0/1199703516 conn(0x7efe5007e370 msgr2=0x7efe50080790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.578+0000 7efe55600700 1 -- 172.21.15.106:0/1199703516 shutdown_connections 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.578+0000 7efe55600700 1 -- 172.21.15.106:0/1199703516 wait complete. 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.579+0000 7efe55600700 1 Processor -- start 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.579+0000 7efe55600700 1 -- start start 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.580+0000 7efe55600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe501a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.580+0000 7efe55600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efe501136b0 con 0x7efe5010fc50 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.580+0000 7efe4ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe501a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.580+0000 7efe4ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe501a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37428/0 (socket says 172.21.15.106:37428) 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.580+0000 7efe4ea00700 1 -- 172.21.15.106:0/3127717486 learned_addr learned my addr 172.21.15.106:0/3127717486 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.580+0000 7efe4ea00700 1 -- 172.21.15.106:0/3127717486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe501a37c0 con 0x7efe5010fc50 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.581+0000 7efe4ea00700 1 --2- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe501a3280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efe40011ac0 tx=0x7efe40011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.581+0000 7efe47400700 1 -- 172.21.15.106:0/3127717486 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe40003970 con 0x7efe5010fc50 2024-04-28T12:40:13.678 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.581+0000 7efe47400700 1 -- 172.21.15.106:0/3127717486 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe40003b10 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.581+0000 7efe47400700 1 -- 172.21.15.106:0/3127717486 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe40024610 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.581+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe501a3a50 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.581+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe501a6740 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.582+0000 7efe47400700 1 -- 172.21.15.106:0/3127717486 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7efe4001a9e0 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.582+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe50064a80 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.583+0000 7efe47400700 1 --2- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7efe34039560 0x7efe3403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.583+0000 7efe47400700 1 -- 172.21.15.106:0/3127717486 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efe4004d8d0 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.583+0000 7efe47e00700 1 --2- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7efe34039560 0x7efe3403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.583+0000 7efe47e00700 1 --2- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7efe34039560 0x7efe3403ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe38006fd0 tx=0x7efe38006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.588+0000 7efe47400700 1 -- 172.21.15.106:0/3127717486 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efe4000b3e0 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.877+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7efe501a6910 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.931+0000 7efe47400700 1 -- 172.21.15.106:0/3127717486 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7efe501a6910 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.935+0000 7efe47400700 1 -- 172.21.15.106:0/3127717486 <== mon.0 v2:172.21.15.106:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7efe4002c350 con 0x7efe5010fc50 2024-04-28T12:40:13.679 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.935+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7efe34039560 msgr2=0x7efe3403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.935+0000 7efe55600700 1 --2- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7efe34039560 0x7efe3403ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe38006fd0 tx=0x7efe38006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.935+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 msgr2=0x7efe501a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.935+0000 7efe55600700 1 --2- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe501a3280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efe40011ac0 tx=0x7efe40011ba0 comp rx=0 tx=0).stop 2024-04-28T12:40:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.935+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 shutdown_connections 2024-04-28T12:40:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.935+0000 7efe55600700 1 --2- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7efe34039560 0x7efe3403ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.935+0000 7efe55600700 1 --2- 172.21.15.106:0/3127717486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe5010fc50 0x7efe501a3280 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.935+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 >> 172.21.15.106:0/3127717486 conn(0x7efe5007e370 msgr2=0x7efe5007eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.936+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 shutdown_connections 2024-04-28T12:40:13.680 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:12.936+0000 7efe55600700 1 -- 172.21.15.106:0/3127717486 wait complete. 2024-04-28T12:40:13.937 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:13.594+0000 7fbaf854e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-28T12:40:14.230 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:13 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3127717486' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-28T12:40:14.230 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:13 smithi106 ceph-mon[28081]: mgrmap e5: x(active, since 5s) 2024-04-28T12:40:14.230 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:14 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:14.013+0000 7fbaf854e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-28T12:40:14.231 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:14 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:14.124+0000 7fbaf854e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-28T12:40:15.284 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:14 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2501621111' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-28T12:40:15.307 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout { 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout } 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.281+0000 7f35fd000700 1 Processor -- start 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.281+0000 7f35fd000700 1 -- start start 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.281+0000 7f35fd000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f8109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.282+0000 7f35fd000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35f8109700 con 0x7f35f8108d10 2024-04-28T12:40:15.308 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.282+0000 7f35f5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f8109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:15.309 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.282+0000 7f35f5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f8109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37464/0 (socket says 172.21.15.106:37464) 2024-04-28T12:40:15.309 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.282+0000 7f35f5e00700 1 -- 172.21.15.106:0/3033132022 learned_addr learned my addr 172.21.15.106:0/3033132022 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:15.309 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.282+0000 7f35f5e00700 1 -- 172.21.15.106:0/3033132022 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f8109f10 con 0x7f35f8108d10 2024-04-28T12:40:15.309 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.283+0000 7f35f5e00700 1 --2- 172.21.15.106:0/3033132022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f8109130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35e80098a0 tx=0x7f35e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7a478559f3b28e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:15.309 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.283+0000 7f35f4a00700 1 -- 172.21.15.106:0/3033132022 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e8004160 con 0x7f35f8108d10 2024-04-28T12:40:15.314 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.283+0000 7f35f4a00700 1 -- 172.21.15.106:0/3033132022 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f35e8004300 con 0x7f35f8108d10 2024-04-28T12:40:15.314 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.283+0000 7f35f4a00700 1 -- 172.21.15.106:0/3033132022 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e8011af0 con 0x7f35f8108d10 2024-04-28T12:40:15.314 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.284+0000 7f35fd000700 1 -- 172.21.15.106:0/3033132022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 msgr2=0x7f35f8109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:15.314 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.284+0000 7f35fd000700 1 --2- 172.21.15.106:0/3033132022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f8109130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f35e80098a0 tx=0x7f35e8009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:15.314 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.284+0000 7f35fd000700 1 -- 172.21.15.106:0/3033132022 shutdown_connections 2024-04-28T12:40:15.314 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.284+0000 7f35fd000700 1 --2- 172.21.15.106:0/3033132022 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f8109130 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:15.314 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.284+0000 7f35fd000700 1 -- 172.21.15.106:0/3033132022 >> 172.21.15.106:0/3033132022 conn(0x7f35f81042f0 msgr2=0x7f35f8106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:15.314 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.284+0000 7f35fd000700 1 -- 172.21.15.106:0/3033132022 shutdown_connections 2024-04-28T12:40:15.314 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.284+0000 7f35fd000700 1 -- 172.21.15.106:0/3033132022 wait complete. 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.285+0000 7f35fd000700 1 Processor -- start 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.286+0000 7f35fd000700 1 -- start start 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.286+0000 7f35fd000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.286+0000 7f35fd000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35f810d2c0 con 0x7f35f8108d10 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.286+0000 7f35f5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.286+0000 7f35f5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37470/0 (socket says 172.21.15.106:37470) 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.286+0000 7f35f5e00700 1 -- 172.21.15.106:0/2501621111 learned_addr learned my addr 172.21.15.106:0/2501621111 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.287+0000 7f35f5e00700 1 -- 172.21.15.106:0/2501621111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f819d230 con 0x7f35f8108d10 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.287+0000 7f35f5e00700 1 --2- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f819ccf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f35e80117b0 tx=0x7f35e8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.287+0000 7f35eea00700 1 -- 172.21.15.106:0/2501621111 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e8003af0 con 0x7f35f8108d10 2024-04-28T12:40:15.315 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.287+0000 7f35eea00700 1 -- 172.21.15.106:0/2501621111 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f35e8020070 con 0x7f35f8108d10 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.287+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f819d4c0 con 0x7f35f8108d10 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.287+0000 7f35eea00700 1 -- 172.21.15.106:0/2501621111 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35e8019630 con 0x7f35f8108d10 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.288+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f81a01a0 con 0x7f35f8108d10 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.289+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f8064a80 con 0x7f35f8108d10 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.289+0000 7f35eea00700 1 -- 172.21.15.106:0/2501621111 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f35e801a430 con 0x7f35f8108d10 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.289+0000 7f35eea00700 1 --2- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f35dc039600 0x7f35dc03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.289+0000 7f35eea00700 1 -- 172.21.15.106:0/2501621111 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f35e804dac0 con 0x7f35f8108d10 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.290+0000 7f35f5400700 1 -- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f35dc039600 msgr2=0x7f35dc03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/102356115 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.290+0000 7f35f5400700 1 --2- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f35dc039600 0x7f35dc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.295+0000 7f35eea00700 1 -- 172.21.15.106:0/2501621111 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f35e80042a0 con 0x7f35f8108d10 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.491+0000 7f35f5400700 1 -- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f35dc039600 msgr2=0x7f35dc03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/102356115 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.491+0000 7f35f5400700 1 --2- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f35dc039600 0x7f35dc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T12:40:15.316 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.582+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f35f81a0370 con 0x7f35f8108d10 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.583+0000 7f35eea00700 1 -- 172.21.15.106:0/2501621111 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f35f81a0370 con 0x7f35f8108d10 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f35dc039600 msgr2=0x7f35dc03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 --2- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f35dc039600 0x7f35dc03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 msgr2=0x7f35f819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 --2- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f819ccf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f35e80117b0 tx=0x7f35e8011b70 comp rx=0 tx=0).stop 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 shutdown_connections 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 --2- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f35dc039600 0x7f35dc03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 --2- 172.21.15.106:0/2501621111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35f8108d10 0x7f35f819ccf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 >> 172.21.15.106:0/2501621111 conn(0x7f35f81042f0 msgr2=0x7f35f8104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 shutdown_connections 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:14.586+0000 7f35fd000700 1 -- 172.21.15.106:0/2501621111 wait complete. 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:Waiting for the mgr to restart... 2024-04-28T12:40:15.317 INFO:teuthology.orchestra.run.smithi106.stdout:Waiting for mgr epoch 5... 2024-04-28T12:40:15.868 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:15 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:15.681+0000 7fbaf854e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-28T12:40:16.123 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:15 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:15.914+0000 7fbaf854e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-28T12:40:16.479 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:16 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:16.126+0000 7fbaf854e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-28T12:40:16.824 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:16 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:16.498+0000 7fbaf854e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-28T12:40:16.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:16 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:16.618+0000 7fbaf854e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-28T12:40:17.115 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:16 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:16.868+0000 7fbaf854e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-28T12:40:17.574 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:17 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:17.118+0000 7fbaf854e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:40:18.324 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:17 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:17.896+0000 7fbaf854e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-28T12:40:18.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:18 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:18.442+0000 7fbaf854e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-28T12:40:18.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:18 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:18.573+0000 7fbaf854e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-28T12:40:19.325 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:18 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:18.838+0000 7fbaf854e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-28T12:40:19.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:19.500+0000 7fbaf854e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-28T12:40:19.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:19.623+0000 7fbaf854e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-28T12:40:20.247 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:19.944+0000 7fbaf854e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-28T12:40:20.247 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:20.120+0000 7fbaf854e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-28T12:40:20.247 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:20.250+0000 7fbaf854e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-28T12:40:20.574 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:20.401+0000 7fbaf854e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-28T12:40:21.074 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:20.770+0000 7fbaf854e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-28T12:40:21.805 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:21.524+0000 7fbaf854e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-28T12:40:22.075 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:21.808+0000 7fbaf854e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-28T12:40:22.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: Active manager daemon x restarted 2024-04-28T12:40:22.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: Activating manager daemon x 2024-04-28T12:40:22.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: osdmap e2: 0 total, 0 up, 0 in 2024-04-28T12:40:22.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: mgrmap e6: x(active, starting, since 0.00871878s) 2024-04-28T12:40:22.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:40:22.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-28T12:40:22.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-28T12:40:22.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-28T12:40:22.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-28T12:40:22.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: Manager daemon x is now available 2024-04-28T12:40:22.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:21 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:23.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:22 smithi106 ceph-mon[28081]: Found migration_current of "None". Setting to last migration. 2024-04-28T12:40:23.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:22 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:23.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:22 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:40:23.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:22 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:40:23.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:22 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-28T12:40:23.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:22 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-28T12:40:23.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:22 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:23.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:22 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:23.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:22 smithi106 ceph-mon[28081]: mgrmap e7: x(active, since 1.01497s) 2024-04-28T12:40:23.483 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout { 2024-04-28T12:40:23.484 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-28T12:40:23.484 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-28T12:40:23.484 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout } 2024-04-28T12:40:23.484 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.871+0000 7f5d51600700 1 Processor -- start 2024-04-28T12:40:23.484 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.871+0000 7f5d51600700 1 -- start start 2024-04-28T12:40:23.484 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.872+0000 7f5d51600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c07d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:23.484 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.872+0000 7f5d51600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c07d850 con 0x7f5d4c07ee20 2024-04-28T12:40:23.484 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.872+0000 7f5d4aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c07d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:23.484 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.873+0000 7f5d4aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c07d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37476/0 (socket says 172.21.15.106:37476) 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.873+0000 7f5d4aa00700 1 -- 172.21.15.106:0/449630341 learned_addr learned my addr 172.21.15.106:0/449630341 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.873+0000 7f5d4aa00700 1 -- 172.21.15.106:0/449630341 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c07d9d0 con 0x7f5d4c07ee20 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.873+0000 7f5d4aa00700 1 --2- 172.21.15.106:0/449630341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c07d280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0098a0 tx=0x7f5d3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25c1ab55050c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.874+0000 7f5d49600700 1 -- 172.21.15.106:0/449630341 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c004160 con 0x7f5d4c07ee20 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.874+0000 7f5d49600700 1 -- 172.21.15.106:0/449630341 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5d3c004300 con 0x7f5d4c07ee20 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.874+0000 7f5d49600700 1 -- 172.21.15.106:0/449630341 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c011af0 con 0x7f5d4c07ee20 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.874+0000 7f5d51600700 1 -- 172.21.15.106:0/449630341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 msgr2=0x7f5d4c07d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.874+0000 7f5d51600700 1 --2- 172.21.15.106:0/449630341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c07d280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0098a0 tx=0x7f5d3c009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.874+0000 7f5d51600700 1 -- 172.21.15.106:0/449630341 shutdown_connections 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.874+0000 7f5d51600700 1 --2- 172.21.15.106:0/449630341 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c07d280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.874+0000 7f5d51600700 1 -- 172.21.15.106:0/449630341 >> 172.21.15.106:0/449630341 conn(0x7f5d4c1042f0 msgr2=0x7f5d4c106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:23.485 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.875+0000 7f5d51600700 1 -- 172.21.15.106:0/449630341 shutdown_connections 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.875+0000 7f5d51600700 1 -- 172.21.15.106:0/449630341 wait complete. 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.876+0000 7f5d51600700 1 Processor -- start 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.876+0000 7f5d51600700 1 -- start start 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.876+0000 7f5d51600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c1a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.876+0000 7f5d51600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4c10aeb0 con 0x7f5d4c07ee20 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.876+0000 7f5d4aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c1a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.876+0000 7f5d4aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c1a32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37482/0 (socket says 172.21.15.106:37482) 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.876+0000 7f5d4aa00700 1 -- 172.21.15.106:0/266104232 learned_addr learned my addr 172.21.15.106:0/266104232 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.877+0000 7f5d4aa00700 1 -- 172.21.15.106:0/266104232 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4c1a37f0 con 0x7f5d4c07ee20 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.877+0000 7f5d4aa00700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c1a32b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0117b0 tx=0x7f5d3c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.877+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c003af0 con 0x7f5d4c07ee20 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.877+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5d3c020070 con 0x7f5d4c07ee20 2024-04-28T12:40:23.486 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.877+0000 7f5d51600700 1 -- 172.21.15.106:0/266104232 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d4c1a3a80 con 0x7f5d4c07ee20 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.877+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d3c019630 con 0x7f5d4c07ee20 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.878+0000 7f5d51600700 1 -- 172.21.15.106:0/266104232 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d4c1a6770 con 0x7f5d4c07ee20 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.879+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f5d3c0198a0 con 0x7f5d4c07ee20 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.879+0000 7f5d43400700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 0x7f5d3003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.879+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 --> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5d3003c1d0 con 0x7f5d30039600 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.879+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5d3c04db40 con 0x7f5d4c07ee20 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.880+0000 7f5d4a000700 1 -- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 msgr2=0x7f5d3003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/102356115 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:15.880+0000 7f5d4a000700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 0x7f5d3003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:16.081+0000 7f5d4a000700 1 -- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 msgr2=0x7f5d3003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/102356115 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:16.081+0000 7f5d4a000700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 0x7f5d3003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:16.482+0000 7f5d4a000700 1 -- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 msgr2=0x7f5d3003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/102356115 2024-04-28T12:40:23.487 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:16.482+0000 7f5d4a000700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 0x7f5d3003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:17.283+0000 7f5d4a000700 1 -- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 msgr2=0x7f5d3003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/102356115 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:17.283+0000 7f5d4a000700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 0x7f5d3003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:18.886+0000 7f5d4a000700 1 -- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 msgr2=0x7f5d3003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/102356115 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:18.886+0000 7f5d4a000700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 0x7f5d3003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:21.818+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f5d3c00b430 con 0x7f5d4c07ee20 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:21.818+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 msgr2=0x7f5d3003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:21.818+0000 7f5d43400700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 0x7f5d3003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.822+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f5d3c04f170 con 0x7f5d4c07ee20 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.822+0000 7f5d43400700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5d3003cef0 0x7f5d3003f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.822+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5d3c01ab90 con 0x7f5d3003cef0 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.823+0000 7f5d4a000700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5d3003cef0 0x7f5d3003f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.825+0000 7f5d4a000700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5d3003cef0 0x7f5d3003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d38003a10 tx=0x7f5d380092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.827+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 <== mgr.14120 v2:172.21.15.106:6800/2253669851 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5d3c01ab90 con 0x7f5d3003cef0 2024-04-28T12:40:23.488 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.837+0000 7f5d51600700 1 -- 172.21.15.106:0/266104232 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5d2c002630 con 0x7f5d3003cef0 2024-04-28T12:40:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.837+0000 7f5d43400700 1 -- 172.21.15.106:0/266104232 <== mgr.14120 v2:172.21.15.106:6800/2253669851 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5d2c002630 con 0x7f5d3003cef0 2024-04-28T12:40:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 -- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5d3003cef0 msgr2=0x7f5d3003f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5d3003cef0 0x7f5d3003f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5d38003a10 tx=0x7f5d380092b0 comp rx=0 tx=0).stop 2024-04-28T12:40:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 -- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 msgr2=0x7f5d4c1a32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c1a32b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0117b0 tx=0x7f5d3c011b70 comp rx=0 tx=0).stop 2024-04-28T12:40:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 -- 172.21.15.106:0/266104232 shutdown_connections 2024-04-28T12:40:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5d3003cef0 0x7f5d3003f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d4c07ee20 0x7f5d4c1a32b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:23.489 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 --2- 172.21.15.106:0/266104232 >> [v2:172.21.15.106:6800/102356115,v1:172.21.15.106:6801/102356115] conn(0x7f5d30039600 0x7f5d3003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:23.490 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 -- 172.21.15.106:0/266104232 >> 172.21.15.106:0/266104232 conn(0x7f5d4c1042f0 msgr2=0x7f5d4c1066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:23.490 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 -- 172.21.15.106:0/266104232 shutdown_connections 2024-04-28T12:40:23.490 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:22.838+0000 7f5d51600700 1 -- 172.21.15.106:0/266104232 wait complete. 2024-04-28T12:40:23.490 INFO:teuthology.orchestra.run.smithi106.stdout:mgr epoch 5 is available 2024-04-28T12:40:23.490 INFO:teuthology.orchestra.run.smithi106.stdout:Setting orchestrator backend to cephadm... 2024-04-28T12:40:24.023 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:23 smithi106 ceph-mon[28081]: [28/Apr/2024:12:40:22] ENGINE Bus STARTING 2024-04-28T12:40:24.024 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:23 smithi106 ceph-mon[28081]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-28T12:40:24.024 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:23 smithi106 ceph-mon[28081]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-28T12:40:24.024 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:23 smithi106 ceph-mon[28081]: [28/Apr/2024:12:40:22] ENGINE Serving on https://172.21.15.106:7150 2024-04-28T12:40:24.024 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:23 smithi106 ceph-mon[28081]: [28/Apr/2024:12:40:22] ENGINE Serving on http://172.21.15.106:8765 2024-04-28T12:40:24.024 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:23 smithi106 ceph-mon[28081]: [28/Apr/2024:12:40:22] ENGINE Bus STARTED 2024-04-28T12:40:24.024 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:23 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:40:25.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:24 smithi106 ceph-mon[28081]: mgrmap e8: x(active, since 2s) 2024-04-28T12:40:25.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:24 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:25.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:24 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:40:25.156 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.181+0000 7fb447400700 1 Processor -- start 2024-04-28T12:40:25.163 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.181+0000 7fb447400700 1 -- start start 2024-04-28T12:40:25.163 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.181+0000 7fb447400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb440112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:25.163 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.181+0000 7fb447400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb440112580 con 0x7fb44010fc50 2024-04-28T12:40:25.163 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.182+0000 7fb444a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb440112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:25.163 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.182+0000 7fb444a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb440112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33556/0 (socket says 172.21.15.106:33556) 2024-04-28T12:40:25.163 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.182+0000 7fb444a00700 1 -- 172.21.15.106:0/3367540151 learned_addr learned my addr 172.21.15.106:0/3367540151 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:25.163 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.182+0000 7fb444a00700 1 -- 172.21.15.106:0/3367540151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb440112700 con 0x7fb44010fc50 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.183+0000 7fb444a00700 1 --2- 172.21.15.106:0/3367540151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb440112040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4340098a0 tx=0x7fb434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b907f5288edbbed server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.183+0000 7fb43f400700 1 -- 172.21.15.106:0/3367540151 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb434004160 con 0x7fb44010fc50 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.183+0000 7fb43f400700 1 -- 172.21.15.106:0/3367540151 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb434004300 con 0x7fb44010fc50 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.183+0000 7fb43f400700 1 -- 172.21.15.106:0/3367540151 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb434011af0 con 0x7fb44010fc50 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.184+0000 7fb447400700 1 -- 172.21.15.106:0/3367540151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 msgr2=0x7fb440112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.184+0000 7fb447400700 1 --2- 172.21.15.106:0/3367540151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb440112040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4340098a0 tx=0x7fb434009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.184+0000 7fb447400700 1 -- 172.21.15.106:0/3367540151 shutdown_connections 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.184+0000 7fb447400700 1 --2- 172.21.15.106:0/3367540151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb440112040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.184+0000 7fb447400700 1 -- 172.21.15.106:0/3367540151 >> 172.21.15.106:0/3367540151 conn(0x7fb44007e370 msgr2=0x7fb440080790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.184+0000 7fb447400700 1 -- 172.21.15.106:0/3367540151 shutdown_connections 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.184+0000 7fb447400700 1 -- 172.21.15.106:0/3367540151 wait complete. 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.185+0000 7fb447400700 1 Processor -- start 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.185+0000 7fb447400700 1 -- start start 2024-04-28T12:40:25.164 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.186+0000 7fb447400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb4401a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.186+0000 7fb447400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb4401136b0 con 0x7fb44010fc50 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.186+0000 7fb444a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb4401a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.186+0000 7fb444a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb4401a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33564/0 (socket says 172.21.15.106:33564) 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.186+0000 7fb444a00700 1 -- 172.21.15.106:0/1147292880 learned_addr learned my addr 172.21.15.106:0/1147292880 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.186+0000 7fb444a00700 1 -- 172.21.15.106:0/1147292880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4401a37c0 con 0x7fb44010fc50 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.186+0000 7fb444a00700 1 --2- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb4401a3280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb434011ac0 tx=0x7fb434011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.187+0000 7fb43d600700 1 -- 172.21.15.106:0/1147292880 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb434003970 con 0x7fb44010fc50 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.187+0000 7fb43d600700 1 -- 172.21.15.106:0/1147292880 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb434003b10 con 0x7fb44010fc50 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.187+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4401a3a50 con 0x7fb44010fc50 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.187+0000 7fb43d600700 1 -- 172.21.15.106:0/1147292880 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb434024610 con 0x7fb44010fc50 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.187+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4401a6740 con 0x7fb44010fc50 2024-04-28T12:40:25.165 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.188+0000 7fb43d600700 1 -- 172.21.15.106:0/1147292880 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb43401a9e0 con 0x7fb44010fc50 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.188+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb440064a80 con 0x7fb44010fc50 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.189+0000 7fb43d600700 1 --2- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fb428039600 0x7fb42803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.189+0000 7fb43d600700 1 -- 172.21.15.106:0/1147292880 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb43404d9a0 con 0x7fb44010fc50 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.189+0000 7fb43fe00700 1 --2- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fb428039600 0x7fb42803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.190+0000 7fb43fe00700 1 --2- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fb428039600 0x7fb42803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006fd0 tx=0x7fb42c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.195+0000 7fb43d600700 1 -- 172.21.15.106:0/1147292880 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb43400b3e0 con 0x7fb44010fc50 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.440+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fb44007f5b0 con 0x7fb428039600 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.449+0000 7fb43d600700 1 -- 172.21.15.106:0/1147292880 <== mgr.14120 v2:172.21.15.106:6800/2253669851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb44007f5b0 con 0x7fb428039600 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fb428039600 msgr2=0x7fb42803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 --2- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fb428039600 0x7fb42803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb42c006fd0 tx=0x7fb42c006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 msgr2=0x7fb4401a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:25.166 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 --2- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb4401a3280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb434011ac0 tx=0x7fb434011ba0 comp rx=0 tx=0).stop 2024-04-28T12:40:25.167 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 shutdown_connections 2024-04-28T12:40:25.167 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 --2- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fb428039600 0x7fb42803bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:25.167 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 --2- 172.21.15.106:0/1147292880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb44010fc50 0x7fb4401a3280 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:25.167 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 >> 172.21.15.106:0/1147292880 conn(0x7fb44007e370 msgr2=0x7fb44007eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:25.167 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 shutdown_connections 2024-04-28T12:40:25.167 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:24.452+0000 7fb447400700 1 -- 172.21.15.106:0/1147292880 wait complete. 2024-04-28T12:40:26.036 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:25 smithi106 ceph-mon[28081]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:40:26.665 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-28T12:40:26.665 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.775+0000 7f45a8800700 1 Processor -- start 2024-04-28T12:40:26.665 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.775+0000 7f45a8800700 1 -- start start 2024-04-28T12:40:26.666 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.775+0000 7f45a8800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a0112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:26.668 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.775+0000 7f45a8800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f45a0112580 con 0x7f45a010fc50 2024-04-28T12:40:26.668 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.776+0000 7f45a5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a0112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.776+0000 7f45a5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a0112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33574/0 (socket says 172.21.15.106:33574) 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.776+0000 7f45a5e00700 1 -- 172.21.15.106:0/615174244 learned_addr learned my addr 172.21.15.106:0/615174244 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.776+0000 7f45a5e00700 1 -- 172.21.15.106:0/615174244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a0112700 con 0x7f45a010fc50 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.776+0000 7f45a5e00700 1 --2- 172.21.15.106:0/615174244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a0112040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45940098a0 tx=0x7f4594009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2e553a9b04a246 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.777+0000 7f45a4a00700 1 -- 172.21.15.106:0/615174244 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4594004160 con 0x7f45a010fc50 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.777+0000 7f45a4a00700 1 -- 172.21.15.106:0/615174244 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4594004300 con 0x7f45a010fc50 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.777+0000 7f45a4a00700 1 -- 172.21.15.106:0/615174244 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4594011af0 con 0x7f45a010fc50 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.777+0000 7f45a8800700 1 -- 172.21.15.106:0/615174244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 msgr2=0x7f45a0112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.777+0000 7f45a8800700 1 --2- 172.21.15.106:0/615174244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a0112040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45940098a0 tx=0x7f4594009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.778+0000 7f45a8800700 1 -- 172.21.15.106:0/615174244 shutdown_connections 2024-04-28T12:40:26.669 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.778+0000 7f45a8800700 1 --2- 172.21.15.106:0/615174244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a0112040 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:26.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.778+0000 7f45a8800700 1 -- 172.21.15.106:0/615174244 >> 172.21.15.106:0/615174244 conn(0x7f45a007e370 msgr2=0x7f45a0080790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:26.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.778+0000 7f45a8800700 1 -- 172.21.15.106:0/615174244 shutdown_connections 2024-04-28T12:40:26.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.778+0000 7f45a8800700 1 -- 172.21.15.106:0/615174244 wait complete. 2024-04-28T12:40:26.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.779+0000 7f45a8800700 1 Processor -- start 2024-04-28T12:40:26.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.779+0000 7f45a8800700 1 -- start start 2024-04-28T12:40:26.670 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.780+0000 7f45a8800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a01a32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:26.673 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.780+0000 7f45a8800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f45a01136b0 con 0x7f45a010fc50 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.780+0000 7f45a5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a01a32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.780+0000 7f45a5e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a01a32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33588/0 (socket says 172.21.15.106:33588) 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.780+0000 7f45a5e00700 1 -- 172.21.15.106:0/3808839398 learned_addr learned my addr 172.21.15.106:0/3808839398 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.781+0000 7f45a5e00700 1 -- 172.21.15.106:0/3808839398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a01a37f0 con 0x7f45a010fc50 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.781+0000 7f45a5e00700 1 --2- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a01a32b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4594011ac0 tx=0x7f4594011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.781+0000 7f459aa00700 1 -- 172.21.15.106:0/3808839398 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4594003970 con 0x7f45a010fc50 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.781+0000 7f459aa00700 1 -- 172.21.15.106:0/3808839398 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4594003b10 con 0x7f45a010fc50 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.781+0000 7f459aa00700 1 -- 172.21.15.106:0/3808839398 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4594024610 con 0x7f45a010fc50 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.781+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a01a3a80 con 0x7f45a010fc50 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.781+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a01a6770 con 0x7f45a010fc50 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.782+0000 7f459aa00700 1 -- 172.21.15.106:0/3808839398 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f459401a9e0 con 0x7f45a010fc50 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.782+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a0064a80 con 0x7f45a010fc50 2024-04-28T12:40:26.674 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.783+0000 7f459aa00700 1 --2- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4588039600 0x7f458803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.783+0000 7f459aa00700 1 -- 172.21.15.106:0/3808839398 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f459404d940 con 0x7f45a010fc50 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.783+0000 7f45a5400700 1 --2- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4588039600 0x7f458803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.784+0000 7f45a5400700 1 --2- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4588039600 0x7f458803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4590006fd0 tx=0x7f4590006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:25.789+0000 7f459aa00700 1 -- 172.21.15.106:0/3808839398 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f459400b3e0 con 0x7f45a010fc50 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.016+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f45a007f590 con 0x7f4588039600 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.018+0000 7f459aa00700 1 -- 172.21.15.106:0/3808839398 <== mgr.14120 v2:172.21.15.106:6800/2253669851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f45a007f590 con 0x7f4588039600 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.021+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4588039600 msgr2=0x7f458803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.021+0000 7f45a8800700 1 --2- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4588039600 0x7f458803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4590006fd0 tx=0x7f4590006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.021+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 msgr2=0x7f45a01a32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.021+0000 7f45a8800700 1 --2- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a01a32b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4594011ac0 tx=0x7f4594011ba0 comp rx=0 tx=0).stop 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.021+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 shutdown_connections 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.021+0000 7f45a8800700 1 --2- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4588039600 0x7f458803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.021+0000 7f45a8800700 1 --2- 172.21.15.106:0/3808839398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45a010fc50 0x7f45a01a32b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:26.675 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.021+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 >> 172.21.15.106:0/3808839398 conn(0x7f45a007e370 msgr2=0x7f45a007ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:26.676 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.022+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 shutdown_connections 2024-04-28T12:40:26.676 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:26.022+0000 7f45a8800700 1 -- 172.21.15.106:0/3808839398 wait complete. 2024-04-28T12:40:26.676 INFO:teuthology.orchestra.run.smithi106.stdout:Generating ssh key... 2024-04-28T12:40:26.953 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:26 smithi106 ceph-mon[28081]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:40:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-mon[28081]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:40:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-mon[28081]: Generating ssh key... 2024-04-28T12:40:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:28.326 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: Generating public/private rsa key pair. 2024-04-28T12:40:28.326 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: Your identification has been saved in /tmp/tmpe0yqb8kx/key. 2024-04-28T12:40:28.326 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: Your public key has been saved in /tmp/tmpe0yqb8kx/key.pub. 2024-04-28T12:40:28.326 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: The key fingerprint is: 2024-04-28T12:40:28.326 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: SHA256:/nEUjFCfA+4wDyvp9+j0sRAiwMvuAxJc5fZK83qE7RI ceph-e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:40:28.326 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: The key's randomart image is: 2024-04-28T12:40:28.326 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: +---[RSA 3072]----+ 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: | .. ..o | 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: | . .. o = . | 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: |. + o + o * | 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: |.o o. .. B o | 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: | .o .o*.S o . | 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: |o. .E+* . . | 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: |... .=.= o . | 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: | .. ..= * = | 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: | .. .o.o = | 2024-04-28T12:40:28.327 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: +----[SHA256]-----+ 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.261+0000 7f147e400700 1 Processor -- start 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.261+0000 7f147e400700 1 -- start start 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.262+0000 7f147e400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147810b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.262+0000 7f147e400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f147810b980 con 0x7f147810af90 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.262+0000 7f1477400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147810b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.262+0000 7f1477400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147810b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33594/0 (socket says 172.21.15.106:33594) 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.262+0000 7f1477400700 1 -- 172.21.15.106:0/316450843 learned_addr learned my addr 172.21.15.106:0/316450843 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.263+0000 7f1477400700 1 -- 172.21.15.106:0/316450843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147810c190 con 0x7f147810af90 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.263+0000 7f1477400700 1 --2- 172.21.15.106:0/316450843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147810b3b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14680098a0 tx=0x7f1468009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc5575dd6c11dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:28.757 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.263+0000 7f1476000700 1 -- 172.21.15.106:0/316450843 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1468004160 con 0x7f147810af90 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.263+0000 7f1476000700 1 -- 172.21.15.106:0/316450843 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1468004300 con 0x7f147810af90 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.263+0000 7f1476000700 1 -- 172.21.15.106:0/316450843 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1468011af0 con 0x7f147810af90 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.264+0000 7f147e400700 1 -- 172.21.15.106:0/316450843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 msgr2=0x7f147810b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.264+0000 7f147e400700 1 --2- 172.21.15.106:0/316450843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147810b3b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f14680098a0 tx=0x7f1468009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.265+0000 7f147e400700 1 -- 172.21.15.106:0/316450843 shutdown_connections 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.265+0000 7f147e400700 1 --2- 172.21.15.106:0/316450843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147810b3b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.265+0000 7f147e400700 1 -- 172.21.15.106:0/316450843 >> 172.21.15.106:0/316450843 conn(0x7f147807e370 msgr2=0x7f147807e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.265+0000 7f147e400700 1 -- 172.21.15.106:0/316450843 shutdown_connections 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.265+0000 7f147e400700 1 -- 172.21.15.106:0/316450843 wait complete. 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.266+0000 7f147e400700 1 Processor -- start 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.266+0000 7f147e400700 1 -- start start 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.266+0000 7f147e400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.266+0000 7f147e400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f147810f3c0 con 0x7f147810af90 2024-04-28T12:40:28.758 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.267+0000 7f1477400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.267+0000 7f1477400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33608/0 (socket says 172.21.15.106:33608) 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.267+0000 7f1477400700 1 -- 172.21.15.106:0/775318235 learned_addr learned my addr 172.21.15.106:0/775318235 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.267+0000 7f1477400700 1 -- 172.21.15.106:0/775318235 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147819f400 con 0x7f147810af90 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.268+0000 7f1477400700 1 --2- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147819eec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f14680117b0 tx=0x7f1468011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.268+0000 7f146fe00700 1 -- 172.21.15.106:0/775318235 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1468003af0 con 0x7f147810af90 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.268+0000 7f146fe00700 1 -- 172.21.15.106:0/775318235 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1468020070 con 0x7f147810af90 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.268+0000 7f146fe00700 1 -- 172.21.15.106:0/775318235 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1468019630 con 0x7f147810af90 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.268+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147819f690 con 0x7f147810af90 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.268+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14781a2380 con 0x7f147810af90 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.269+0000 7f146fe00700 1 -- 172.21.15.106:0/775318235 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f146801a9c0 con 0x7f147810af90 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.269+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f147810f540 con 0x7f147810af90 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.270+0000 7f146fe00700 1 --2- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f145c0395b0 0x7f145c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:28.759 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.270+0000 7f146fe00700 1 -- 172.21.15.106:0/775318235 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f146804db40 con 0x7f147810af90 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.270+0000 7f1476a00700 1 --2- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f145c0395b0 0x7f145c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.271+0000 7f1476a00700 1 --2- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f145c0395b0 0x7f145c03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1464006fd0 tx=0x7f1464006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.275+0000 7f146fe00700 1 -- 172.21.15.106:0/775318235 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f146800e260 con 0x7f147810af90 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:27.503+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1478108b30 con 0x7f145c0395b0 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.078+0000 7f146fe00700 1 -- 172.21.15.106:0/775318235 <== mgr.14120 v2:172.21.15.106:6800/2253669851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1478108b30 con 0x7f145c0395b0 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f145c0395b0 msgr2=0x7f145c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 --2- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f145c0395b0 0x7f145c03ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1464006fd0 tx=0x7f1464006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 msgr2=0x7f147819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 --2- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147819eec0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f14680117b0 tx=0x7f1468011b70 comp rx=0 tx=0).stop 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 shutdown_connections 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 --2- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f145c0395b0 0x7f145c03ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 --2- 172.21.15.106:0/775318235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f147810af90 0x7f147819eec0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 >> 172.21.15.106:0/775318235 conn(0x7f147807e370 msgr2=0x7f1478108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 shutdown_connections 2024-04-28T12:40:28.760 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:28.082+0000 7f147e400700 1 -- 172.21.15.106:0/775318235 wait complete. 2024-04-28T12:40:30.181 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:30 smithi106 ceph-mon[28081]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:40:30.208 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKlBDNJmjY05MfSLHae6C/aVTAyDhX09M6wCgVXgTAhJPi5e2y/c41gywq+YGdsPUoHb9AramtQFFrnecYfyNz53eGt9FvraqKn7YKBXKx/9fl8i2D6hbH0faDeyu7uAwe8zQ/6mzLsC+wTqNTNr6r87P34I26WYObqOCDPPHZ4DV20hhLa8uVK9VWHlh+wBJr5uY1WQGEB3ZmZCRtmkqu8MB+qakkk1cVRxwvcSqfQCd1DZYDq4fS+DGBo+7Dr2/VcVwezJcv1SK6J3SUzHI0k5ncpKRLwBclyPt9yPnc35IzNluBb9cEwv3bjlmdchhfTU9COQ4dOzjfPUp4O7lAR6g80YZy30F3Z8GwtT+ayJkR3T1Bg/1NzWF9ohwkFusX33+vADtYqCVYLxZ4QWiqlkHCB9fhWNTrczc/83upSqihY3j2MbvFlExVmQou4YSZsvcYeQtoBFGe4xhN368V/dD5cFveOCdKzHUru025fqFQWMcDtGxrCksVkCH+OUU= ceph-e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:40:30.208 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.308+0000 7f1c18a00700 1 Processor -- start 2024-04-28T12:40:30.208 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.308+0000 7f1c18a00700 1 -- start start 2024-04-28T12:40:30.208 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.309+0000 7f1c18a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1410b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:30.208 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.309+0000 7f1c18a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1410b860 con 0x7f1c1410af00 2024-04-28T12:40:30.208 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.309+0000 7f1c11e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1410b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.309+0000 7f1c11e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1410b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54954/0 (socket says 172.21.15.106:54954) 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.309+0000 7f1c11e00700 1 -- 172.21.15.106:0/2670457957 learned_addr learned my addr 172.21.15.106:0/2670457957 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.310+0000 7f1c11e00700 1 -- 172.21.15.106:0/2670457957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1410c060 con 0x7f1c1410af00 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.310+0000 7f1c11e00700 1 --2- 172.21.15.106:0/2670457957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1410b320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1c040098a0 tx=0x7f1c04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebfedf35a3f268d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.310+0000 7f1c10a00700 1 -- 172.21.15.106:0/2670457957 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c04004160 con 0x7f1c1410af00 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.310+0000 7f1c10a00700 1 -- 172.21.15.106:0/2670457957 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1c04004300 con 0x7f1c1410af00 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.311+0000 7f1c10a00700 1 -- 172.21.15.106:0/2670457957 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c04011af0 con 0x7f1c1410af00 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.311+0000 7f1c18a00700 1 -- 172.21.15.106:0/2670457957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 msgr2=0x7f1c1410b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.311+0000 7f1c18a00700 1 --2- 172.21.15.106:0/2670457957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1410b320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1c040098a0 tx=0x7f1c04009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.311+0000 7f1c18a00700 1 -- 172.21.15.106:0/2670457957 shutdown_connections 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.311+0000 7f1c18a00700 1 --2- 172.21.15.106:0/2670457957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1410b320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.311+0000 7f1c18a00700 1 -- 172.21.15.106:0/2670457957 >> 172.21.15.106:0/2670457957 conn(0x7f1c1407e2a0 msgr2=0x7f1c1407e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.311+0000 7f1c18a00700 1 -- 172.21.15.106:0/2670457957 shutdown_connections 2024-04-28T12:40:30.209 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.311+0000 7f1c18a00700 1 -- 172.21.15.106:0/2670457957 wait complete. 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.312+0000 7f1c18a00700 1 Processor -- start 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.313+0000 7f1c18a00700 1 -- start start 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.313+0000 7f1c18a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.313+0000 7f1c18a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1410f4a0 con 0x7f1c1410af00 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.313+0000 7f1c11e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.313+0000 7f1c11e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54964/0 (socket says 172.21.15.106:54964) 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.313+0000 7f1c11e00700 1 -- 172.21.15.106:0/1445676029 learned_addr learned my addr 172.21.15.106:0/1445676029 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.314+0000 7f1c11e00700 1 -- 172.21.15.106:0/1445676029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1419f3f0 con 0x7f1c1410af00 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.314+0000 7f1c11e00700 1 --2- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1419eeb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c04011ac0 tx=0x7f1c04011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.314+0000 7f1c0aa00700 1 -- 172.21.15.106:0/1445676029 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c04003970 con 0x7f1c1410af00 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.314+0000 7f1c0aa00700 1 -- 172.21.15.106:0/1445676029 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1c04003b10 con 0x7f1c1410af00 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.314+0000 7f1c0aa00700 1 -- 172.21.15.106:0/1445676029 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c04024610 con 0x7f1c1410af00 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.314+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1419f680 con 0x7f1c1410af00 2024-04-28T12:40:30.210 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.314+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c141a2370 con 0x7f1c1410af00 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.316+0000 7f1c0aa00700 1 -- 172.21.15.106:0/1445676029 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1c0401a9e0 con 0x7f1c1410af00 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.316+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c14064a80 con 0x7f1c1410af00 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.316+0000 7f1c0aa00700 1 --2- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f1bf8039600 0x7f1bf803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.316+0000 7f1c0aa00700 1 -- 172.21.15.106:0/1445676029 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1c0404d980 con 0x7f1c1410af00 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.316+0000 7f1c11400700 1 --2- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f1bf8039600 0x7f1bf803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.317+0000 7f1c11400700 1 --2- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f1bf8039600 0x7f1bf803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1c00006fd0 tx=0x7f1c00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.321+0000 7f1c0aa00700 1 -- 172.21.15.106:0/1445676029 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1c0400b3e0 con 0x7f1c1410af00 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.549+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f1c14108b30 con 0x7f1bf8039600 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.551+0000 7f1c0aa00700 1 -- 172.21.15.106:0/1445676029 <== mgr.14120 v2:172.21.15.106:6800/2253669851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f1c14108b30 con 0x7f1bf8039600 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.554+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f1bf8039600 msgr2=0x7f1bf803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.554+0000 7f1c18a00700 1 --2- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f1bf8039600 0x7f1bf803bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1c00006fd0 tx=0x7f1c00006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.554+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 msgr2=0x7f1c1419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:30.211 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.554+0000 7f1c18a00700 1 --2- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1419eeb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c04011ac0 tx=0x7f1c04011ba0 comp rx=0 tx=0).stop 2024-04-28T12:40:30.212 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.555+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 shutdown_connections 2024-04-28T12:40:30.212 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.555+0000 7f1c18a00700 1 --2- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f1bf8039600 0x7f1bf803bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:30.212 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.555+0000 7f1c18a00700 1 --2- 172.21.15.106:0/1445676029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c1410af00 0x7f1c1419eeb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:30.212 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.555+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 >> 172.21.15.106:0/1445676029 conn(0x7f1c1407e2a0 msgr2=0x7f1c14108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:30.212 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.555+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 shutdown_connections 2024-04-28T12:40:30.212 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:29.555+0000 7f1c18a00700 1 -- 172.21.15.106:0/1445676029 wait complete. 2024-04-28T12:40:30.212 INFO:teuthology.orchestra.run.smithi106.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-28T12:40:30.212 INFO:teuthology.orchestra.run.smithi106.stdout:Adding key to root@localhost authorized_keys... 2024-04-28T12:40:30.212 INFO:teuthology.orchestra.run.smithi106.stdout:Adding host smithi106... 2024-04-28T12:40:32.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:32 smithi106 ceph-mon[28081]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi106", "addr": "172.21.15.106", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:40:33.206 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:33 smithi106 ceph-mon[28081]: Deploying cephadm binary to smithi106 2024-04-28T12:40:37.868 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout Added host 'smithi106' with addr '172.21.15.106' 2024-04-28T12:40:37.868 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.818+0000 7f698d000700 1 Processor -- start 2024-04-28T12:40:37.868 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.819+0000 7f698d000700 1 -- start start 2024-04-28T12:40:37.868 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.819+0000 7f698d000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f6988112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:37.868 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.819+0000 7f698d000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112580 con 0x7f698810fc50 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.819+0000 7f6985e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f6988112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.820+0000 7f6985e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f6988112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54972/0 (socket says 172.21.15.106:54972) 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.820+0000 7f6985e00700 1 -- 172.21.15.106:0/3800613957 learned_addr learned my addr 172.21.15.106:0/3800613957 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.820+0000 7f6985e00700 1 -- 172.21.15.106:0/3800613957 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6988112700 con 0x7f698810fc50 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.820+0000 7f6985e00700 1 --2- 172.21.15.106:0/3800613957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f6988112040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f69780098a0 tx=0x7f6978009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670db9e8d01b68a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.821+0000 7f6984a00700 1 -- 172.21.15.106:0/3800613957 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6978004160 con 0x7f698810fc50 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.821+0000 7f6984a00700 1 -- 172.21.15.106:0/3800613957 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6978004300 con 0x7f698810fc50 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.821+0000 7f6984a00700 1 -- 172.21.15.106:0/3800613957 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6978011af0 con 0x7f698810fc50 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.821+0000 7f698d000700 1 -- 172.21.15.106:0/3800613957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 msgr2=0x7f6988112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.821+0000 7f698d000700 1 --2- 172.21.15.106:0/3800613957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f6988112040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f69780098a0 tx=0x7f6978009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.821+0000 7f698d000700 1 -- 172.21.15.106:0/3800613957 shutdown_connections 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.821+0000 7f698d000700 1 --2- 172.21.15.106:0/3800613957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f6988112040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.822+0000 7f698d000700 1 -- 172.21.15.106:0/3800613957 >> 172.21.15.106:0/3800613957 conn(0x7f698807e370 msgr2=0x7f6988080790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:37.869 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.822+0000 7f698d000700 1 -- 172.21.15.106:0/3800613957 shutdown_connections 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.822+0000 7f698d000700 1 -- 172.21.15.106:0/3800613957 wait complete. 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.823+0000 7f698d000700 1 Processor -- start 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.823+0000 7f698d000700 1 -- start start 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.823+0000 7f698d000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f69881a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.823+0000 7f698d000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f69881136b0 con 0x7f698810fc50 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.823+0000 7f6985e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f69881a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.823+0000 7f6985e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f69881a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54984/0 (socket says 172.21.15.106:54984) 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.823+0000 7f6985e00700 1 -- 172.21.15.106:0/3742465271 learned_addr learned my addr 172.21.15.106:0/3742465271 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.824+0000 7f6985e00700 1 -- 172.21.15.106:0/3742465271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69881a37c0 con 0x7f698810fc50 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.824+0000 7f6985e00700 1 --2- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f69881a3280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6978011ac0 tx=0x7f6978011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.824+0000 7f697ea00700 1 -- 172.21.15.106:0/3742465271 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6978003970 con 0x7f698810fc50 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.824+0000 7f697ea00700 1 -- 172.21.15.106:0/3742465271 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6978003b10 con 0x7f698810fc50 2024-04-28T12:40:37.870 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.824+0000 7f697ea00700 1 -- 172.21.15.106:0/3742465271 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6978024610 con 0x7f698810fc50 2024-04-28T12:40:37.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.824+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a3a50 con 0x7f698810fc50 2024-04-28T12:40:37.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.825+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a6740 con 0x7f698810fc50 2024-04-28T12:40:37.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.826+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6988064a80 con 0x7f698810fc50 2024-04-28T12:40:37.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.826+0000 7f697ea00700 1 -- 172.21.15.106:0/3742465271 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f697801a9e0 con 0x7f698810fc50 2024-04-28T12:40:37.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.826+0000 7f697ea00700 1 --2- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f696c039600 0x7f696c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:37.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.826+0000 7f697ea00700 1 -- 172.21.15.106:0/3742465271 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f697804d980 con 0x7f698810fc50 2024-04-28T12:40:37.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.827+0000 7f6985400700 1 --2- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f696c039600 0x7f696c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:37.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.827+0000 7f6985400700 1 --2- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f696c039600 0x7f696c03bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6974006fd0 tx=0x7f6974006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:37.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:30.832+0000 7f697ea00700 1 -- 172.21.15.106:0/3742465271 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f697800b3e0 con 0x7f698810fc50 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:31.066+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi106", "addr": "172.21.15.106", "target": ["mon-mgr", ""]}) -- 0x7f698807f5b0 con 0x7f696c039600 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.204+0000 7f697ea00700 1 -- 172.21.15.106:0/3742465271 <== mgr.14120 v2:172.21.15.106:6800/2253669851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f698807f5b0 con 0x7f696c039600 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.207+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f696c039600 msgr2=0x7f696c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.207+0000 7f698d000700 1 --2- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f696c039600 0x7f696c03bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6974006fd0 tx=0x7f6974006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.207+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 msgr2=0x7f69881a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.207+0000 7f698d000700 1 --2- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f69881a3280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6978011ac0 tx=0x7f6978011ba0 comp rx=0 tx=0).stop 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.207+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 shutdown_connections 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.207+0000 7f698d000700 1 --2- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f696c039600 0x7f696c03bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.207+0000 7f698d000700 1 --2- 172.21.15.106:0/3742465271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f698810fc50 0x7f69881a3280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.207+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 >> 172.21.15.106:0/3742465271 conn(0x7f698807e370 msgr2=0x7f698807eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.208+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 shutdown_connections 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:37.208+0000 7f698d000700 1 -- 172.21.15.106:0/3742465271 wait complete. 2024-04-28T12:40:37.872 INFO:teuthology.orchestra.run.smithi106.stdout:Deploying unmanaged mon service... 2024-04-28T12:40:38.495 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:38 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:38.495 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:38 smithi106 ceph-mon[28081]: Added host smithi106 2024-04-28T12:40:38.495 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:38 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:40:39.766 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-28T12:40:39.766 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.498+0000 7f4e37e00700 1 Processor -- start 2024-04-28T12:40:39.766 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.498+0000 7f4e37e00700 1 -- start start 2024-04-28T12:40:39.766 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.498+0000 7f4e37e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e38112040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.766 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.498+0000 7f4e37e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4e380833f0 con 0x7f4e3810fc50 2024-04-28T12:40:39.766 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.499+0000 7f4e36a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e38112040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.499+0000 7f4e36a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e38112040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54996/0 (socket says 172.21.15.106:54996) 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.499+0000 7f4e36a00700 1 -- 172.21.15.106:0/3331991096 learned_addr learned my addr 172.21.15.106:0/3331991096 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.499+0000 7f4e36a00700 1 -- 172.21.15.106:0/3331991096 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e38112580 con 0x7f4e3810fc50 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.500+0000 7f4e36a00700 1 --2- 172.21.15.106:0/3331991096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e38112040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e280098a0 tx=0x7f4e28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3ab72b12b6a3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.500+0000 7f4e35600700 1 -- 172.21.15.106:0/3331991096 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e28004160 con 0x7f4e3810fc50 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.500+0000 7f4e35600700 1 -- 172.21.15.106:0/3331991096 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e28004300 con 0x7f4e3810fc50 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.500+0000 7f4e35600700 1 -- 172.21.15.106:0/3331991096 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e28011af0 con 0x7f4e3810fc50 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.501+0000 7f4e37e00700 1 -- 172.21.15.106:0/3331991096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 msgr2=0x7f4e38112040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.501+0000 7f4e37e00700 1 --2- 172.21.15.106:0/3331991096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e38112040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e280098a0 tx=0x7f4e28009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.501+0000 7f4e37e00700 1 -- 172.21.15.106:0/3331991096 shutdown_connections 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.501+0000 7f4e37e00700 1 --2- 172.21.15.106:0/3331991096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e38112040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.501+0000 7f4e37e00700 1 -- 172.21.15.106:0/3331991096 >> 172.21.15.106:0/3331991096 conn(0x7f4e3807e170 msgr2=0x7f4e38080590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.501+0000 7f4e37e00700 1 -- 172.21.15.106:0/3331991096 shutdown_connections 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.501+0000 7f4e37e00700 1 -- 172.21.15.106:0/3331991096 wait complete. 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.502+0000 7f4e37e00700 1 Processor -- start 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.502+0000 7f4e37e00700 1 -- start start 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.502+0000 7f4e37e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e381a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.502+0000 7f4e37e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4e38113480 con 0x7f4e3810fc50 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.502+0000 7f4e36a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e381a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.503+0000 7f4e36a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e381a3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:55004/0 (socket says 172.21.15.106:55004) 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.503+0000 7f4e36a00700 1 -- 172.21.15.106:0/498928004 learned_addr learned my addr 172.21.15.106:0/498928004 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.503+0000 7f4e36a00700 1 -- 172.21.15.106:0/498928004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e381a35c0 con 0x7f4e3810fc50 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.503+0000 7f4e36a00700 1 --2- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e381a3080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e28011ac0 tx=0x7f4e28011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.503+0000 7f4e3ca00700 1 -- 172.21.15.106:0/498928004 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e28003970 con 0x7f4e3810fc50 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.503+0000 7f4e3ca00700 1 -- 172.21.15.106:0/498928004 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4e28003b10 con 0x7f4e3810fc50 2024-04-28T12:40:39.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.503+0000 7f4e3ca00700 1 -- 172.21.15.106:0/498928004 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e28024610 con 0x7f4e3810fc50 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.503+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e381a3850 con 0x7f4e3810fc50 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.503+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e381a6540 con 0x7f4e3810fc50 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.504+0000 7f4e3ca00700 1 -- 172.21.15.106:0/498928004 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4e2801a9e0 con 0x7f4e3810fc50 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.504+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e38064a80 con 0x7f4e3810fc50 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.505+0000 7f4e3ca00700 1 --2- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4e1c039600 0x7f4e1c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.505+0000 7f4e3ca00700 1 -- 172.21.15.106:0/498928004 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4e2804d980 con 0x7f4e3810fc50 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.505+0000 7f4e36000700 1 --2- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4e1c039600 0x7f4e1c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.506+0000 7f4e36000700 1 --2- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4e1c039600 0x7f4e1c03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e24006fd0 tx=0x7f4e24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.510+0000 7f4e3ca00700 1 -- 172.21.15.106:0/498928004 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4e2800b3e0 con 0x7f4e3810fc50 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.749+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4e3807f370 con 0x7f4e1c039600 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.757+0000 7f4e3ca00700 1 -- 172.21.15.106:0/498928004 <== mgr.14120 v2:172.21.15.106:6800/2253669851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4e3807f370 con 0x7f4e1c039600 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4e1c039600 msgr2=0x7f4e1c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:39.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 --2- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4e1c039600 0x7f4e1c03bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e24006fd0 tx=0x7f4e24006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:39.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 msgr2=0x7f4e381a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:39.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 --2- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e381a3080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4e28011ac0 tx=0x7f4e28011ba0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 shutdown_connections 2024-04-28T12:40:39.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 --2- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f4e1c039600 0x7f4e1c03bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 --2- 172.21.15.106:0/498928004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4e3810fc50 0x7f4e381a3080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:39.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 >> 172.21.15.106:0/498928004 conn(0x7f4e3807e170 msgr2=0x7f4e3807ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:39.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 shutdown_connections 2024-04-28T12:40:39.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:38.760+0000 7f4e37e00700 1 -- 172.21.15.106:0/498928004 wait complete. 2024-04-28T12:40:39.775 INFO:teuthology.orchestra.run.smithi106.stdout:Deploying unmanaged mgr service... 2024-04-28T12:40:40.036 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:39 smithi106 ceph-mon[28081]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:40:40.036 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:39 smithi106 ceph-mon[28081]: Saving service mon spec with placement count:5 2024-04-28T12:40:40.036 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:39 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:41.767 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.726+0000 7fafd6600700 1 Processor -- start 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.726+0000 7fafd6600700 1 -- start start 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.727+0000 7fafd6600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd007d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.727+0000 7fafd6600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fafd007ef70 con 0x7fafd007d240 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.727+0000 7fafd5200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd007d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.727+0000 7fafd5200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd007d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46022/0 (socket says 172.21.15.106:46022) 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.727+0000 7fafd5200700 1 -- 172.21.15.106:0/1439868993 learned_addr learned my addr 172.21.15.106:0/1439868993 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.727+0000 7fafd5200700 1 -- 172.21.15.106:0/1439868993 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd007dba0 con 0x7fafd007d240 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.728+0000 7fafd5200700 1 --2- 172.21.15.106:0/1439868993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd007d660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009e80 tx=0x7fafc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c259d2ae24dd48c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.728+0000 7fafcf400700 1 -- 172.21.15.106:0/1439868993 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc4017070 con 0x7fafd007d240 2024-04-28T12:40:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.728+0000 7fafcf400700 1 -- 172.21.15.106:0/1439868993 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafc400b8c0 con 0x7fafd007d240 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.728+0000 7fafcf400700 1 -- 172.21.15.106:0/1439868993 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc4004490 con 0x7fafd007d240 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.729+0000 7fafd6600700 1 -- 172.21.15.106:0/1439868993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 msgr2=0x7fafd007d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.729+0000 7fafd6600700 1 --2- 172.21.15.106:0/1439868993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd007d660 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fafc4009e80 tx=0x7fafc400b330 comp rx=0 tx=0).stop 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.729+0000 7fafd6600700 1 -- 172.21.15.106:0/1439868993 shutdown_connections 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.729+0000 7fafd6600700 1 --2- 172.21.15.106:0/1439868993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd007d660 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.729+0000 7fafd6600700 1 -- 172.21.15.106:0/1439868993 >> 172.21.15.106:0/1439868993 conn(0x7fafd0104150 msgr2=0x7fafd0106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.729+0000 7fafd6600700 1 -- 172.21.15.106:0/1439868993 shutdown_connections 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.729+0000 7fafd6600700 1 -- 172.21.15.106:0/1439868993 wait complete. 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.730+0000 7fafd6600700 1 Processor -- start 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.730+0000 7fafd6600700 1 -- start start 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.730+0000 7fafd6600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.731+0000 7fafd6600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fafd010f220 con 0x7fafd007d240 2024-04-28T12:40:41.769 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.731+0000 7fafd5200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.731+0000 7fafd5200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46030/0 (socket says 172.21.15.106:46030) 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.731+0000 7fafd5200700 1 -- 172.21.15.106:0/3856727155 learned_addr learned my addr 172.21.15.106:0/3856727155 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.731+0000 7fafd5200700 1 -- 172.21.15.106:0/3856727155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd019f290 con 0x7fafd007d240 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.731+0000 7fafd5200700 1 --2- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd019ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fafc400b7b0 tx=0x7fafc400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.732+0000 7fafcd600700 1 -- 172.21.15.106:0/3856727155 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc4020020 con 0x7fafd007d240 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.732+0000 7fafcd600700 1 -- 172.21.15.106:0/3856727155 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fafc4003d10 con 0x7fafd007d240 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.732+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd019f520 con 0x7fafd007d240 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.732+0000 7fafcd600700 1 -- 172.21.15.106:0/3856727155 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafc4017040 con 0x7fafd007d240 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.732+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd01a2210 con 0x7fafd007d240 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.733+0000 7fafcd600700 1 -- 172.21.15.106:0/3856727155 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fafc4023020 con 0x7fafd007d240 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.733+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd0064a80 con 0x7fafd007d240 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.733+0000 7fafcd600700 1 --2- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fafb8039600 0x7fafb803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.733+0000 7fafcd600700 1 -- 172.21.15.106:0/3856727155 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fafc400f050 con 0x7fafd007d240 2024-04-28T12:40:41.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.734+0000 7fafcfe00700 1 --2- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fafb8039600 0x7fafb803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.734+0000 7fafcfe00700 1 --2- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fafb8039600 0x7fafb803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006fd0 tx=0x7fafbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.738+0000 7fafcd600700 1 -- 172.21.15.106:0/3856727155 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fafc402a020 con 0x7fafd007d240 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.764+0000 7fafcd600700 1 -- 172.21.15.106:0/3856727155 <== mon.0 v2:172.21.15.106:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc401b550 con 0x7fafd007d240 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:40.999+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fafd0105350 con 0x7fafb8039600 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.009+0000 7fafcd600700 1 -- 172.21.15.106:0/3856727155 <== mgr.14120 v2:172.21.15.106:6800/2253669851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fafd0105350 con 0x7fafb8039600 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fafb8039600 msgr2=0x7fafb803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 --2- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fafb8039600 0x7fafb803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fafbc006fd0 tx=0x7fafbc006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 msgr2=0x7fafd019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 --2- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd019ed50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fafc400b7b0 tx=0x7fafc400be90 comp rx=0 tx=0).stop 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 shutdown_connections 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 --2- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fafb8039600 0x7fafb803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 --2- 172.21.15.106:0/3856727155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fafd007d240 0x7fafd019ed50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:41.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 >> 172.21.15.106:0/3856727155 conn(0x7fafd0104150 msgr2=0x7fafd0104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:41.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 shutdown_connections 2024-04-28T12:40:41.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:41.012+0000 7fafd6600700 1 -- 172.21.15.106:0/3856727155 wait complete. 2024-04-28T12:40:42.014 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:41 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:42.014 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:41 smithi106 ceph-mon[28081]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:40:42.014 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:41 smithi106 ceph-mon[28081]: Saving service mgr spec with placement count:2 2024-04-28T12:40:42.014 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:41 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:42.014 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:41 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:43.213 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:42 smithi106 ceph-mon[28081]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:40:43.214 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:42 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4034414641' entity='client.admin' 2024-04-28T12:40:43.770 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.298+0000 7fe3e9600700 1 Processor -- start 2024-04-28T12:40:43.771 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.298+0000 7fe3e9600700 1 -- start start 2024-04-28T12:40:43.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.299+0000 7fe3e9600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e410b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:43.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.299+0000 7fe3e9600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e410b860 con 0x7fe3e410af00 2024-04-28T12:40:43.772 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.299+0000 7fe3e2a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e410b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:43.773 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.299+0000 7fe3e2a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e410b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46046/0 (socket says 172.21.15.106:46046) 2024-04-28T12:40:43.774 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.299+0000 7fe3e2a00700 1 -- 172.21.15.106:0/3015971533 learned_addr learned my addr 172.21.15.106:0/3015971533 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:43.774 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.300+0000 7fe3e2a00700 1 -- 172.21.15.106:0/3015971533 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e410c060 con 0x7fe3e410af00 2024-04-28T12:40:43.774 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.300+0000 7fe3e2a00700 1 --2- 172.21.15.106:0/3015971533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e410b320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40098a0 tx=0x7fe3d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3a9c83a50c5009 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:43.774 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.300+0000 7fe3e1600700 1 -- 172.21.15.106:0/3015971533 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3d4004160 con 0x7fe3e410af00 2024-04-28T12:40:43.774 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.300+0000 7fe3e1600700 1 -- 172.21.15.106:0/3015971533 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d4004300 con 0x7fe3e410af00 2024-04-28T12:40:43.774 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.300+0000 7fe3e1600700 1 -- 172.21.15.106:0/3015971533 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3d4011a20 con 0x7fe3e410af00 2024-04-28T12:40:43.774 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.301+0000 7fe3e9600700 1 -- 172.21.15.106:0/3015971533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 msgr2=0x7fe3e410b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:43.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.301+0000 7fe3e9600700 1 --2- 172.21.15.106:0/3015971533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e410b320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40098a0 tx=0x7fe3d4009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:43.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.301+0000 7fe3e9600700 1 -- 172.21.15.106:0/3015971533 shutdown_connections 2024-04-28T12:40:43.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.301+0000 7fe3e9600700 1 --2- 172.21.15.106:0/3015971533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e410b320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:43.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.301+0000 7fe3e9600700 1 -- 172.21.15.106:0/3015971533 >> 172.21.15.106:0/3015971533 conn(0x7fe3e407e2d0 msgr2=0x7fe3e407e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:43.775 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.301+0000 7fe3e9600700 1 -- 172.21.15.106:0/3015971533 shutdown_connections 2024-04-28T12:40:43.776 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.301+0000 7fe3e9600700 1 -- 172.21.15.106:0/3015971533 wait complete. 2024-04-28T12:40:43.776 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.302+0000 7fe3e9600700 1 Processor -- start 2024-04-28T12:40:43.776 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.302+0000 7fe3e9600700 1 -- start start 2024-04-28T12:40:43.777 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.303+0000 7fe3e9600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:43.777 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.303+0000 7fe3e9600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e410f4a0 con 0x7fe3e410af00 2024-04-28T12:40:43.777 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.303+0000 7fe3e2a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:43.777 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.303+0000 7fe3e2a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46062/0 (socket says 172.21.15.106:46062) 2024-04-28T12:40:43.777 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.303+0000 7fe3e2a00700 1 -- 172.21.15.106:0/4034414641 learned_addr learned my addr 172.21.15.106:0/4034414641 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:43.778 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.303+0000 7fe3e2a00700 1 -- 172.21.15.106:0/4034414641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e419f350 con 0x7fe3e410af00 2024-04-28T12:40:43.778 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.304+0000 7fe3e2a00700 1 --2- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e419ee10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4004660 tx=0x7fe3d4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:43.778 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.304+0000 7fe3db400700 1 -- 172.21.15.106:0/4034414641 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3d4003aa0 con 0x7fe3e410af00 2024-04-28T12:40:43.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.304+0000 7fe3db400700 1 -- 172.21.15.106:0/4034414641 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d401b8f0 con 0x7fe3e410af00 2024-04-28T12:40:43.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.304+0000 7fe3db400700 1 -- 172.21.15.106:0/4034414641 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3d4023900 con 0x7fe3e410af00 2024-04-28T12:40:43.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.304+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e419f5e0 con 0x7fe3e410af00 2024-04-28T12:40:43.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.304+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e41a22d0 con 0x7fe3e410af00 2024-04-28T12:40:43.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.305+0000 7fe3db400700 1 -- 172.21.15.106:0/4034414641 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe3d4023b70 con 0x7fe3e410af00 2024-04-28T12:40:43.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.306+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e410f620 con 0x7fe3e410af00 2024-04-28T12:40:43.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.306+0000 7fe3db400700 1 --2- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fe3c8039600 0x7fe3c803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:43.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.306+0000 7fe3db400700 1 -- 172.21.15.106:0/4034414641 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe3d404def0 con 0x7fe3e410af00 2024-04-28T12:40:43.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.306+0000 7fe3e2000700 1 --2- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fe3c8039600 0x7fe3c803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:43.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.307+0000 7fe3e2000700 1 --2- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fe3c8039600 0x7fe3c803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0006fd0 tx=0x7fe3d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:43.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.316+0000 7fe3db400700 1 -- 172.21.15.106:0/4034414641 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe3d401f030 con 0x7fe3e410af00 2024-04-28T12:40:43.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.544+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fe3e41a28b0 con 0x7fe3e410af00 2024-04-28T12:40:43.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.547+0000 7fe3db400700 1 -- 172.21.15.106:0/4034414641 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fe3d404d840 con 0x7fe3e410af00 2024-04-28T12:40:43.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.550+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fe3c8039600 msgr2=0x7fe3c803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:43.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.550+0000 7fe3e9600700 1 --2- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fe3c8039600 0x7fe3c803bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0006fd0 tx=0x7fe3d0006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:43.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.550+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 msgr2=0x7fe3e419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:43.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.550+0000 7fe3e9600700 1 --2- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e419ee10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4004660 tx=0x7fe3d4011aa0 comp rx=0 tx=0).stop 2024-04-28T12:40:43.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.550+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 shutdown_connections 2024-04-28T12:40:43.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.550+0000 7fe3e9600700 1 --2- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fe3c8039600 0x7fe3c803bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:43.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.550+0000 7fe3e9600700 1 --2- 172.21.15.106:0/4034414641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3e410af00 0x7fe3e419ee10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:43.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.550+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 >> 172.21.15.106:0/4034414641 conn(0x7fe3e407e2d0 msgr2=0x7fe3e41083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:43.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.550+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 shutdown_connections 2024-04-28T12:40:43.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:42.551+0000 7fe3e9600700 1 -- 172.21.15.106:0/4034414641 wait complete. 2024-04-28T12:40:44.989 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:44 smithi106 ceph-mon[28081]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:40:44.989 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:44 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:44.989 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:44 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1219335581' entity='client.admin' 2024-04-28T12:40:45.320 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.389+0000 7f943a600700 1 Processor -- start 2024-04-28T12:40:45.320 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.389+0000 7f943a600700 1 -- start start 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.390+0000 7f943a600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f9434109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.390+0000 7f943a600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9434109700 con 0x7f9434108d10 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.390+0000 7f9433400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f9434109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.390+0000 7f9433400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f9434109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46066/0 (socket says 172.21.15.106:46066) 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.390+0000 7f9433400700 1 -- 172.21.15.106:0/2556622319 learned_addr learned my addr 172.21.15.106:0/2556622319 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.391+0000 7f9433400700 1 -- 172.21.15.106:0/2556622319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9434109f10 con 0x7f9434108d10 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.391+0000 7f9433400700 1 --2- 172.21.15.106:0/2556622319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f9434109130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94240098a0 tx=0x7f9424009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da4dd69eef6a54b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.391+0000 7f9432000700 1 -- 172.21.15.106:0/2556622319 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9424004160 con 0x7f9434108d10 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.392+0000 7f9432000700 1 -- 172.21.15.106:0/2556622319 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9424004300 con 0x7f9434108d10 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.392+0000 7f9432000700 1 -- 172.21.15.106:0/2556622319 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9424011a20 con 0x7f9434108d10 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.392+0000 7f943a600700 1 -- 172.21.15.106:0/2556622319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 msgr2=0x7f9434109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:45.321 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.392+0000 7f943a600700 1 --2- 172.21.15.106:0/2556622319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f9434109130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94240098a0 tx=0x7f9424009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.392+0000 7f943a600700 1 -- 172.21.15.106:0/2556622319 shutdown_connections 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.392+0000 7f943a600700 1 --2- 172.21.15.106:0/2556622319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f9434109130 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.392+0000 7f943a600700 1 -- 172.21.15.106:0/2556622319 >> 172.21.15.106:0/2556622319 conn(0x7f94341042f0 msgr2=0x7f9434106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.392+0000 7f943a600700 1 -- 172.21.15.106:0/2556622319 shutdown_connections 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.393+0000 7f943a600700 1 -- 172.21.15.106:0/2556622319 wait complete. 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.393+0000 7f943a600700 1 Processor -- start 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.394+0000 7f943a600700 1 -- start start 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.394+0000 7f943a600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f943419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.394+0000 7f943a600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f943410d2c0 con 0x7f9434108d10 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.394+0000 7f9433400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f943419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.394+0000 7f9433400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f943419cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46082/0 (socket says 172.21.15.106:46082) 2024-04-28T12:40:45.322 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.394+0000 7f9433400700 1 -- 172.21.15.106:0/1219335581 learned_addr learned my addr 172.21.15.106:0/1219335581 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.395+0000 7f9433400700 1 -- 172.21.15.106:0/1219335581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943419d180 con 0x7f9434108d10 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.395+0000 7f9433400700 1 --2- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f943419cc40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9424000c00 tx=0x7f9424011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.395+0000 7f942be00700 1 -- 172.21.15.106:0/1219335581 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9424003aa0 con 0x7f9434108d10 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.395+0000 7f942be00700 1 -- 172.21.15.106:0/1219335581 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f942401b8f0 con 0x7f9434108d10 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.395+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943419d410 con 0x7f9434108d10 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.395+0000 7f942be00700 1 -- 172.21.15.106:0/1219335581 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9424023890 con 0x7f9434108d10 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.395+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94341a0100 con 0x7f9434108d10 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.397+0000 7f942be00700 1 -- 172.21.15.106:0/1219335581 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f942401b440 con 0x7f9434108d10 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.397+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9434064a80 con 0x7f9434108d10 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.397+0000 7f942be00700 1 --2- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f9418039600 0x7f941803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.397+0000 7f942be00700 1 -- 172.21.15.106:0/1219335581 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f942404de00 con 0x7f9434108d10 2024-04-28T12:40:45.323 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.398+0000 7f9432a00700 1 --2- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f9418039600 0x7f941803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.398+0000 7f9432a00700 1 --2- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f9418039600 0x7f941803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9420006fd0 tx=0x7f9420006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.403+0000 7f942be00700 1 -- 172.21.15.106:0/1219335581 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f942401f030 con 0x7f9434108d10 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.630+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f94341a0760 con 0x7f9434108d10 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.633+0000 7f942be00700 1 -- 172.21.15.106:0/1219335581 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f942401bda0 con 0x7f9434108d10 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.636+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f9418039600 msgr2=0x7f941803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.636+0000 7f943a600700 1 --2- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f9418039600 0x7f941803bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9420006fd0 tx=0x7f9420006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.636+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 msgr2=0x7f943419cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.636+0000 7f943a600700 1 --2- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f943419cc40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9424000c00 tx=0x7f9424011aa0 comp rx=0 tx=0).stop 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.636+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 shutdown_connections 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.637+0000 7f943a600700 1 --2- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f9418039600 0x7f941803bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.637+0000 7f943a600700 1 --2- 172.21.15.106:0/1219335581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9434108d10 0x7f943419cc40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.637+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 >> 172.21.15.106:0/1219335581 conn(0x7f94341042f0 msgr2=0x7f9434104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:45.324 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.637+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 shutdown_connections 2024-04-28T12:40:45.325 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:44.637+0000 7f943a600700 1 -- 172.21.15.106:0/1219335581 wait complete. 2024-04-28T12:40:45.325 INFO:teuthology.orchestra.run.smithi106.stdout:Enabling the dashboard module... 2024-04-28T12:40:47.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:46 smithi106 ceph-mon[28081]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:40:47.053 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:46 smithi106 ceph-mon[28081]: from='mgr.14120 172.21.15.106:0/2742954809' entity='mgr.x' 2024-04-28T12:40:47.053 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:46 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/952236473' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-28T12:40:47.310 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: ignoring --setuser ceph since I am not root 2024-04-28T12:40:47.310 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: ignoring --setgroup ceph since I am not root 2024-04-28T12:40:47.310 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:47.273+0000 7f2bcd123200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-28T12:40:47.574 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:47.532+0000 7f2bcd123200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:40:47.947 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:47.677+0000 7f2bcd123200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-28T12:40:48.278 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:47 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/952236473' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-28T12:40:48.278 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:47 smithi106 ceph-mon[28081]: mgrmap e9: x(active, since 25s) 2024-04-28T12:40:48.279 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:48 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:48.084+0000 7f2bcd123200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-28T12:40:48.279 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:48 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:48.195+0000 7f2bcd123200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-28T12:40:48.392 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.893+0000 7fa5f7600700 1 Processor -- start 2024-04-28T12:40:48.392 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.893+0000 7fa5f7600700 1 -- start start 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.893+0000 7fa5f7600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f0109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.893+0000 7fa5f7600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0109700 con 0x7fa5f0108d10 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.894+0000 7fa5f4c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f0109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.894+0000 7fa5f4c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f0109130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46098/0 (socket says 172.21.15.106:46098) 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.894+0000 7fa5f4c00700 1 -- 172.21.15.106:0/662731253 learned_addr learned my addr 172.21.15.106:0/662731253 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.894+0000 7fa5f4c00700 1 -- 172.21.15.106:0/662731253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f0109f10 con 0x7fa5f0108d10 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.895+0000 7fa5f4c00700 1 --2- 172.21.15.106:0/662731253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f0109130 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40098a0 tx=0x7fa5e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc000c33fe4584c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.895+0000 7fa5ef400700 1 -- 172.21.15.106:0/662731253 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5e4004160 con 0x7fa5f0108d10 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.895+0000 7fa5ef400700 1 -- 172.21.15.106:0/662731253 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e4004300 con 0x7fa5f0108d10 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.895+0000 7fa5ef400700 1 -- 172.21.15.106:0/662731253 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5e4011a20 con 0x7fa5f0108d10 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.896+0000 7fa5f7600700 1 -- 172.21.15.106:0/662731253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 msgr2=0x7fa5f0109130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:48.393 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.896+0000 7fa5f7600700 1 --2- 172.21.15.106:0/662731253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f0109130 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40098a0 tx=0x7fa5e4009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.896+0000 7fa5f7600700 1 -- 172.21.15.106:0/662731253 shutdown_connections 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.896+0000 7fa5f7600700 1 --2- 172.21.15.106:0/662731253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f0109130 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.896+0000 7fa5f7600700 1 -- 172.21.15.106:0/662731253 >> 172.21.15.106:0/662731253 conn(0x7fa5f01042f0 msgr2=0x7fa5f0106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.896+0000 7fa5f7600700 1 -- 172.21.15.106:0/662731253 shutdown_connections 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.896+0000 7fa5f7600700 1 -- 172.21.15.106:0/662731253 wait complete. 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.897+0000 7fa5f7600700 1 Processor -- start 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.898+0000 7fa5f7600700 1 -- start start 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.898+0000 7fa5f7600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f007eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.898+0000 7fa5f7600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f010d2c0 con 0x7fa5f0108d10 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.898+0000 7fa5f4c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f007eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.898+0000 7fa5f4c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f007eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46106/0 (socket says 172.21.15.106:46106) 2024-04-28T12:40:48.394 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.898+0000 7fa5f4c00700 1 -- 172.21.15.106:0/952236473 learned_addr learned my addr 172.21.15.106:0/952236473 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:48.395 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.899+0000 7fa5f4c00700 1 -- 172.21.15.106:0/952236473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f007f030 con 0x7fa5f0108d10 2024-04-28T12:40:48.395 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.899+0000 7fa5f4c00700 1 --2- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f007eaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4000c00 tx=0x7fa5e4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:48.395 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.899+0000 7fa5ed600700 1 -- 172.21.15.106:0/952236473 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5e4003aa0 con 0x7fa5f0108d10 2024-04-28T12:40:48.395 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.899+0000 7fa5ed600700 1 -- 172.21.15.106:0/952236473 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5e401b8f0 con 0x7fa5f0108d10 2024-04-28T12:40:48.395 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.899+0000 7fa5ed600700 1 -- 172.21.15.106:0/952236473 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5e4023890 con 0x7fa5f0108d10 2024-04-28T12:40:48.395 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.899+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f007d220 con 0x7fa5f0108d10 2024-04-28T12:40:48.395 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.900+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f007d760 con 0x7fa5f0108d10 2024-04-28T12:40:48.395 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.901+0000 7fa5ed600700 1 -- 172.21.15.106:0/952236473 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa5e402c3f0 con 0x7fa5f0108d10 2024-04-28T12:40:48.395 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.901+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f0064a80 con 0x7fa5f0108d10 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.901+0000 7fa5ed600700 1 --2- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fa5d8039600 0x7fa5d803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.901+0000 7fa5ed600700 1 -- 172.21.15.106:0/952236473 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa5e404de30 con 0x7fa5f0108d10 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.902+0000 7fa5efe00700 1 --2- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fa5d8039600 0x7fa5d803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.902+0000 7fa5efe00700 1 --2- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fa5d8039600 0x7fa5d803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc006fd0 tx=0x7fa5dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:45.907+0000 7fa5ed600700 1 -- 172.21.15.106:0/952236473 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa5e401f030 con 0x7fa5f0108d10 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.222+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa5f007e040 con 0x7fa5f0108d10 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.949+0000 7fa5ed600700 1 -- 172.21.15.106:0/952236473 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa5e40202b0 con 0x7fa5f0108d10 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.956+0000 7fa5ed600700 1 -- 172.21.15.106:0/952236473 <== mon.0 v2:172.21.15.106:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa5e40193d0 con 0x7fa5f0108d10 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fa5d8039600 msgr2=0x7fa5d803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 --2- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fa5d8039600 0x7fa5d803bac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc006fd0 tx=0x7fa5dc006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:48.396 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 msgr2=0x7fa5f007eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:48.398 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 --2- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f007eaf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4000c00 tx=0x7fa5e4011aa0 comp rx=0 tx=0).stop 2024-04-28T12:40:48.398 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 shutdown_connections 2024-04-28T12:40:48.398 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 --2- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7fa5d8039600 0x7fa5d803bac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:48.398 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 --2- 172.21.15.106:0/952236473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5f0108d10 0x7fa5f007eaf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:48.398 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 >> 172.21.15.106:0/952236473 conn(0x7fa5f01042f0 msgr2=0x7fa5f0104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:48.398 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 shutdown_connections 2024-04-28T12:40:48.398 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:46.958+0000 7fa5f7600700 1 -- 172.21.15.106:0/952236473 wait complete. 2024-04-28T12:40:49.571 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:49 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3792114443' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-28T12:40:49.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:49 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:49.774+0000 7f2bcd123200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-28T12:40:49.871 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout { 2024-04-28T12:40:49.873 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout } 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.934+0000 7f5cdbc00700 1 Processor -- start 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.934+0000 7f5cdbc00700 1 -- start start 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.935+0000 7f5cdbc00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd407d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.935+0000 7f5cdbc00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd407d830 con 0x7f5cd407ee00 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.935+0000 7f5cd9200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd407d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.935+0000 7f5cd9200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd407d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43312/0 (socket says 172.21.15.106:43312) 2024-04-28T12:40:49.874 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.935+0000 7f5cd9200700 1 -- 172.21.15.106:0/2628474638 learned_addr learned my addr 172.21.15.106:0/2628474638 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:49.875 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.936+0000 7f5cd9200700 1 -- 172.21.15.106:0/2628474638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd407d9b0 con 0x7f5cd407ee00 2024-04-28T12:40:49.875 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.936+0000 7f5cd9200700 1 --2- 172.21.15.106:0/2628474638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd407d260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80098a0 tx=0x7f5cc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9341e47048f3fc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:49.875 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.936+0000 7f5cd3400700 1 -- 172.21.15.106:0/2628474638 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc8004160 con 0x7f5cd407ee00 2024-04-28T12:40:49.875 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.936+0000 7f5cd3400700 1 -- 172.21.15.106:0/2628474638 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc8004300 con 0x7f5cd407ee00 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.936+0000 7f5cd3400700 1 -- 172.21.15.106:0/2628474638 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc8011a20 con 0x7f5cd407ee00 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.937+0000 7f5cdbc00700 1 -- 172.21.15.106:0/2628474638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 msgr2=0x7f5cd407d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.937+0000 7f5cdbc00700 1 --2- 172.21.15.106:0/2628474638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd407d260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80098a0 tx=0x7f5cc8009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.937+0000 7f5cdbc00700 1 -- 172.21.15.106:0/2628474638 shutdown_connections 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.937+0000 7f5cdbc00700 1 --2- 172.21.15.106:0/2628474638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd407d260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.937+0000 7f5cdbc00700 1 -- 172.21.15.106:0/2628474638 >> 172.21.15.106:0/2628474638 conn(0x7f5cd41042f0 msgr2=0x7f5cd4106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.937+0000 7f5cdbc00700 1 -- 172.21.15.106:0/2628474638 shutdown_connections 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.937+0000 7f5cdbc00700 1 -- 172.21.15.106:0/2628474638 wait complete. 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.938+0000 7f5cdbc00700 1 Processor -- start 2024-04-28T12:40:49.880 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.938+0000 7f5cdbc00700 1 -- start start 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.939+0000 7f5cdbc00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.939+0000 7f5cdbc00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410f330 con 0x7f5cd407ee00 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.939+0000 7f5cd9200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.939+0000 7f5cd9200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43322/0 (socket says 172.21.15.106:43322) 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.939+0000 7f5cd9200700 1 -- 172.21.15.106:0/3792114443 learned_addr learned my addr 172.21.15.106:0/3792114443 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.939+0000 7f5cd9200700 1 -- 172.21.15.106:0/3792114443 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd419f310 con 0x7f5cd407ee00 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.940+0000 7f5cd9200700 1 --2- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd419edd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8004660 tx=0x7f5cc8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.940+0000 7f5cd1600700 1 -- 172.21.15.106:0/3792114443 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc8003aa0 con 0x7f5cd407ee00 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.940+0000 7f5cd1600700 1 -- 172.21.15.106:0/3792114443 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc801b8f0 con 0x7f5cd407ee00 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.940+0000 7f5cd1600700 1 -- 172.21.15.106:0/3792114443 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc8023900 con 0x7f5cd407ee00 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.940+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd419f5a0 con 0x7f5cd407ee00 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.940+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41a2290 con 0x7f5cd407ee00 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.941+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd4064a80 con 0x7f5cd407ee00 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.942+0000 7f5cd1600700 1 -- 172.21.15.106:0/3792114443 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5cc801baa0 con 0x7f5cd407ee00 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.942+0000 7f5cd1600700 1 --2- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5cbc039660 0x7f5cbc03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:49.881 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.942+0000 7f5cd1600700 1 -- 172.21.15.106:0/3792114443 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5cc804df10 con 0x7f5cd407ee00 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.942+0000 7f5cd3e00700 1 -- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5cbc039660 msgr2=0x7f5cbc03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/2253669851 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.942+0000 7f5cd3e00700 1 --2- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5cbc039660 0x7f5cbc03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:48.947+0000 7f5cd1600700 1 -- 172.21.15.106:0/3792114443 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5cc801f030 con 0x7f5cd407ee00 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.143+0000 7f5cd3e00700 1 -- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5cbc039660 msgr2=0x7f5cbc03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/2253669851 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.143+0000 7f5cd3e00700 1 --2- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5cbc039660 0x7f5cbc03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.226+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5cd41a2460 con 0x7f5cd407ee00 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.226+0000 7f5cd1600700 1 -- 172.21.15.106:0/3792114443 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f5cc8006d60 con 0x7f5cd407ee00 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.229+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5cbc039660 msgr2=0x7f5cbc03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.229+0000 7f5cdbc00700 1 --2- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5cbc039660 0x7f5cbc03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.229+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 msgr2=0x7f5cd419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.229+0000 7f5cdbc00700 1 --2- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd419edd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8004660 tx=0x7f5cc8011aa0 comp rx=0 tx=0).stop 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.230+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 shutdown_connections 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.230+0000 7f5cdbc00700 1 --2- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f5cbc039660 0x7f5cbc03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:49.882 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.230+0000 7f5cdbc00700 1 --2- 172.21.15.106:0/3792114443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd407ee00 0x7f5cd419edd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:49.883 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.230+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 >> 172.21.15.106:0/3792114443 conn(0x7f5cd41042f0 msgr2=0x7f5cd4104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:49.883 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.230+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 shutdown_connections 2024-04-28T12:40:49.883 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:49.230+0000 7f5cdbc00700 1 -- 172.21.15.106:0/3792114443 wait complete. 2024-04-28T12:40:49.883 INFO:teuthology.orchestra.run.smithi106.stdout:Waiting for the mgr to restart... 2024-04-28T12:40:49.883 INFO:teuthology.orchestra.run.smithi106.stdout:Waiting for mgr epoch 9... 2024-04-28T12:40:50.202 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:50 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:50.018+0000 7f2bcd123200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-28T12:40:50.509 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:50 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:50.238+0000 7f2bcd123200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-28T12:40:50.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:50 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:50.625+0000 7f2bcd123200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-28T12:40:50.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:50 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:50.747+0000 7f2bcd123200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-28T12:40:51.262 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:51 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:51.001+0000 7f2bcd123200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-28T12:40:51.262 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:51 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:51.265+0000 7f2bcd123200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:40:52.324 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:52 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:52.044+0000 7f2bcd123200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-28T12:40:52.970 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:52 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:52.584+0000 7f2bcd123200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-28T12:40:52.970 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:52 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:52.711+0000 7f2bcd123200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-28T12:40:52.970 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:52 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:52.973+0000 7f2bcd123200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-28T12:40:53.824 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:53.614+0000 7f2bcd123200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-28T12:40:53.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:53.735+0000 7f2bcd123200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-28T12:40:54.324 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:54.054+0000 7f2bcd123200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-28T12:40:54.325 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:54.229+0000 7f2bcd123200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-28T12:40:54.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:54.354+0000 7f2bcd123200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-28T12:40:54.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:54.502+0000 7f2bcd123200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-28T12:40:55.324 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:54.867+0000 7f2bcd123200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-28T12:40:55.896 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:55.614+0000 7f2bcd123200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-28T12:40:56.325 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:40:55.899+0000 7f2bcd123200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-28T12:40:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: Active manager daemon x restarted 2024-04-28T12:40:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: Activating manager daemon x 2024-04-28T12:40:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: osdmap e3: 0 total, 0 up, 0 in 2024-04-28T12:40:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: mgrmap e10: x(active, starting, since 0.00908783s) 2024-04-28T12:40:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:40:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-28T12:40:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-28T12:40:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-28T12:40:56.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-28T12:40:56.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:55 smithi106 ceph-mon[28081]: Manager daemon x is now available 2024-04-28T12:40:57.320 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:40:57.320 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-28T12:40:57.320 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-28T12:40:57.321 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:56 smithi106 ceph-mon[28081]: mgrmap e11: x(active, since 1.01092s) 2024-04-28T12:40:57.778 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout { 2024-04-28T12:40:57.778 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-28T12:40:57.778 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-28T12:40:57.778 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout } 2024-04-28T12:40:57.778 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.512+0000 7f6feb000700 1 Processor -- start 2024-04-28T12:40:57.778 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.512+0000 7f6feb000700 1 -- start start 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.513+0000 7f6feb000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe407d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.513+0000 7f6feb000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe407d850 con 0x7f6fe407ee20 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.513+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe407d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.513+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe407d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43336/0 (socket says 172.21.15.106:43336) 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.514+0000 7f6fe3e00700 1 -- 172.21.15.106:0/2944481106 learned_addr learned my addr 172.21.15.106:0/2944481106 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.514+0000 7f6fe3e00700 1 -- 172.21.15.106:0/2944481106 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe407d9d0 con 0x7f6fe407ee20 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.514+0000 7f6fe3e00700 1 --2- 172.21.15.106:0/2944481106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe407d280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40098a0 tx=0x7f6fd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbfcd349f748506 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.514+0000 7f6fe2a00700 1 -- 172.21.15.106:0/2944481106 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fd4004160 con 0x7f6fe407ee20 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.515+0000 7f6fe2a00700 1 -- 172.21.15.106:0/2944481106 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd4004300 con 0x7f6fe407ee20 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.515+0000 7f6fe2a00700 1 -- 172.21.15.106:0/2944481106 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fd4011a20 con 0x7f6fe407ee20 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.515+0000 7f6feb000700 1 -- 172.21.15.106:0/2944481106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 msgr2=0x7f6fe407d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.515+0000 7f6feb000700 1 --2- 172.21.15.106:0/2944481106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe407d280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40098a0 tx=0x7f6fd4009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.515+0000 7f6feb000700 1 -- 172.21.15.106:0/2944481106 shutdown_connections 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.515+0000 7f6feb000700 1 --2- 172.21.15.106:0/2944481106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe407d280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:57.779 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.515+0000 7f6feb000700 1 -- 172.21.15.106:0/2944481106 >> 172.21.15.106:0/2944481106 conn(0x7f6fe41042f0 msgr2=0x7f6fe4106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.516+0000 7f6feb000700 1 -- 172.21.15.106:0/2944481106 shutdown_connections 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.516+0000 7f6feb000700 1 -- 172.21.15.106:0/2944481106 wait complete. 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.517+0000 7f6feb000700 1 Processor -- start 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.517+0000 7f6feb000700 1 -- start start 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.517+0000 7f6feb000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.517+0000 7f6feb000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe410f310 con 0x7f6fe407ee20 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.517+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.517+0000 7f6fe3e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe419ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43342/0 (socket says 172.21.15.106:43342) 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.517+0000 7f6fe3e00700 1 -- 172.21.15.106:0/1040797800 learned_addr learned my addr 172.21.15.106:0/1040797800 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.518+0000 7f6fe3e00700 1 -- 172.21.15.106:0/1040797800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe419f360 con 0x7f6fe407ee20 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.518+0000 7f6fe3e00700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe419ee20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000c00 tx=0x7f6fd4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.518+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fd4003aa0 con 0x7f6fe407ee20 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.518+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd401b8f0 con 0x7f6fe407ee20 2024-04-28T12:40:57.780 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.518+0000 7f6feb000700 1 -- 172.21.15.106:0/1040797800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe419f5f0 con 0x7f6fe407ee20 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.519+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fd4023890 con 0x7f6fe407ee20 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.519+0000 7f6feb000700 1 -- 172.21.15.106:0/1040797800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe41a22e0 con 0x7f6fe407ee20 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.520+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6fd401b440 con 0x7f6fe407ee20 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.520+0000 7f6fe0c00700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 0x7f6fc803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.520+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 --> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6fc803c1d0 con 0x7f6fc8039600 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.520+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6fd404deb0 con 0x7f6fe407ee20 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.520+0000 7f6fe3400700 1 -- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 msgr2=0x7f6fc803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/2253669851 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.520+0000 7f6fe3400700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 0x7f6fc803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.721+0000 7f6fe3400700 1 -- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 msgr2=0x7f6fc803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/2253669851 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:50.721+0000 7f6fe3400700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 0x7f6fc803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:51.122+0000 7f6fe3400700 1 -- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 msgr2=0x7f6fc803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/2253669851 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:51.122+0000 7f6fe3400700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 0x7f6fc803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:51.923+0000 7f6fe3400700 1 -- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 msgr2=0x7f6fc803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/2253669851 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:51.923+0000 7f6fe3400700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 0x7f6fc803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-28T12:40:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:53.525+0000 7f6fe3400700 1 -- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 msgr2=0x7f6fc803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.106:6800/2253669851 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:53.525+0000 7f6fe3400700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 0x7f6fc803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:55.909+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f6fd402c770 con 0x7f6fe407ee20 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:55.909+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 msgr2=0x7f6fc803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:55.909+0000 7f6fe0c00700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 0x7f6fc803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.910+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f6fd4021d40 con 0x7f6fe407ee20 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.910+0000 7f6fe0c00700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6fc803cfe0 0x7f6fc803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.910+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6fc803c1d0 con 0x7f6fc803cfe0 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.911+0000 7f6fe3400700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6fc803cfe0 0x7f6fc803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.913+0000 7f6fe3400700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6fc803cfe0 0x7f6fc803f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0003a10 tx=0x7f6fd00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.914+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f6fc803c1d0 con 0x7f6fc803cfe0 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.929+0000 7f6feb000700 1 -- 172.21.15.106:0/1040797800 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6fe4064a80 con 0x7f6fc803cfe0 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.930+0000 7f6fe0c00700 1 -- 172.21.15.106:0/1040797800 <== mgr.14152 v2:172.21.15.106:6800/1556498064 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6fe4064a80 con 0x7f6fc803cfe0 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 -- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6fc803cfe0 msgr2=0x7f6fc803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6fc803cfe0 0x7f6fc803f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0003a10 tx=0x7f6fd00092b0 comp rx=0 tx=0).stop 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 -- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 msgr2=0x7f6fe419ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe419ee20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4000c00 tx=0x7f6fd4011aa0 comp rx=0 tx=0).stop 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 -- 172.21.15.106:0/1040797800 shutdown_connections 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6fc803cfe0 0x7f6fc803f3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe407ee20 0x7f6fe419ee20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 --2- 172.21.15.106:0/1040797800 >> [v2:172.21.15.106:6800/2253669851,v1:172.21.15.106:6801/2253669851] conn(0x7f6fc8039600 0x7f6fc803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 -- 172.21.15.106:0/1040797800 >> 172.21.15.106:0/1040797800 conn(0x7f6fe41042f0 msgr2=0x7f6fe4104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 -- 172.21.15.106:0/1040797800 shutdown_connections 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:56.931+0000 7f6feb000700 1 -- 172.21.15.106:0/1040797800 wait complete. 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:mgr epoch 9 is available 2024-04-28T12:40:57.783 INFO:teuthology.orchestra.run.smithi106.stdout:Generating a dashboard self-signed certificate... 2024-04-28T12:40:58.034 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:57 smithi106 ceph-mon[28081]: [28/Apr/2024:12:40:57] ENGINE Bus STARTING 2024-04-28T12:40:59.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:58 smithi106 ceph-mon[28081]: [28/Apr/2024:12:40:57] ENGINE Serving on https://172.21.15.106:7150 2024-04-28T12:40:59.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:58 smithi106 ceph-mon[28081]: [28/Apr/2024:12:40:57] ENGINE Serving on http://172.21.15.106:8765 2024-04-28T12:40:59.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:58 smithi106 ceph-mon[28081]: [28/Apr/2024:12:40:57] ENGINE Bus STARTED 2024-04-28T12:40:59.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:58 smithi106 ceph-mon[28081]: mgrmap e12: x(active, since 2s) 2024-04-28T12:40:59.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.784+0000 7f6b52000700 1 Processor -- start 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.785+0000 7f6b52000700 1 -- start start 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.785+0000 7f6b52000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c07d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.785+0000 7f6b52000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c07d890 con 0x7f6b4c07ee60 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.786+0000 7f6b4b400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c07d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.786+0000 7f6b4b400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c07d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43422/0 (socket says 172.21.15.106:43422) 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.786+0000 7f6b4b400700 1 -- 172.21.15.106:0/710819020 learned_addr learned my addr 172.21.15.106:0/710819020 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.786+0000 7f6b4b400700 1 -- 172.21.15.106:0/710819020 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c07da10 con 0x7f6b4c07ee60 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.787+0000 7f6b4b400700 1 --2- 172.21.15.106:0/710819020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c07d2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0098a0 tx=0x7f6b3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b7ad6b04d8a984 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:59.783 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.787+0000 7f6b4a000700 1 -- 172.21.15.106:0/710819020 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b3c004160 con 0x7f6b4c07ee60 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.787+0000 7f6b4a000700 1 -- 172.21.15.106:0/710819020 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3c004300 con 0x7f6b4c07ee60 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.787+0000 7f6b4a000700 1 -- 172.21.15.106:0/710819020 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b3c011a20 con 0x7f6b4c07ee60 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.788+0000 7f6b52000700 1 -- 172.21.15.106:0/710819020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 msgr2=0x7f6b4c07d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.788+0000 7f6b52000700 1 --2- 172.21.15.106:0/710819020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c07d2c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0098a0 tx=0x7f6b3c009bb0 comp rx=0 tx=0).stop 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.788+0000 7f6b52000700 1 -- 172.21.15.106:0/710819020 shutdown_connections 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.788+0000 7f6b52000700 1 --2- 172.21.15.106:0/710819020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c07d2c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.788+0000 7f6b52000700 1 -- 172.21.15.106:0/710819020 >> 172.21.15.106:0/710819020 conn(0x7f6b4c1042a0 msgr2=0x7f6b4c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.789+0000 7f6b52000700 1 -- 172.21.15.106:0/710819020 shutdown_connections 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.789+0000 7f6b52000700 1 -- 172.21.15.106:0/710819020 wait complete. 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.790+0000 7f6b52000700 1 Processor -- start 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.790+0000 7f6b52000700 1 -- start start 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.790+0000 7f6b52000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c1a3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.790+0000 7f6b52000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c10aee0 con 0x7f6b4c07ee60 2024-04-28T12:40:59.784 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.790+0000 7f6b4b400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c1a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.791+0000 7f6b4b400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c1a3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53220/0 (socket says 172.21.15.106:53220) 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.791+0000 7f6b4b400700 1 -- 172.21.15.106:0/1116457397 learned_addr learned my addr 172.21.15.106:0/1116457397 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.791+0000 7f6b4b400700 1 -- 172.21.15.106:0/1116457397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c1a3780 con 0x7f6b4c07ee60 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.791+0000 7f6b4b400700 1 --2- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c1a3240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009520 tx=0x7f6b3c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.792+0000 7f6b43e00700 1 -- 172.21.15.106:0/1116457397 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b3c0038b0 con 0x7f6b4c07ee60 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.792+0000 7f6b43e00700 1 -- 172.21.15.106:0/1116457397 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b3c003a50 con 0x7f6b4c07ee60 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.792+0000 7f6b52000700 1 -- 172.21.15.106:0/1116457397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4c1a3a10 con 0x7f6b4c07ee60 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.792+0000 7f6b43e00700 1 -- 172.21.15.106:0/1116457397 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b3c019690 con 0x7f6b4c07ee60 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.792+0000 7f6b52000700 1 -- 172.21.15.106:0/1116457397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b4c1a6700 con 0x7f6b4c07ee60 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.793+0000 7f6b43e00700 1 -- 172.21.15.106:0/1116457397 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f6b3c01a430 con 0x7f6b4c07ee60 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.793+0000 7f6b41600700 1 -- 172.21.15.106:0/1116457397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b24005330 con 0x7f6b4c07ee60 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.794+0000 7f6b43e00700 1 --2- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6b30039650 0x7f6b3003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.794+0000 7f6b43e00700 1 -- 172.21.15.106:0/1116457397 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6b3c04de60 con 0x7f6b4c07ee60 2024-04-28T12:40:59.785 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.794+0000 7f6b4aa00700 1 --2- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6b30039650 0x7f6b3003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.795+0000 7f6b4aa00700 1 --2- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6b30039650 0x7f6b3003bb10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b38006fd0 tx=0x7f6b38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:58.800+0000 7f6b43e00700 1 -- 172.21.15.106:0/1116457397 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6b3c019e20 con 0x7f6b4c07ee60 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.035+0000 7f6b41600700 1 -- 172.21.15.106:0/1116457397 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6b24000bc0 con 0x7f6b30039650 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.138+0000 7f6b43e00700 1 -- 172.21.15.106:0/1116457397 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6b24000bc0 con 0x7f6b30039650 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.140+0000 7f6b41600700 1 -- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6b30039650 msgr2=0x7f6b3003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.140+0000 7f6b41600700 1 --2- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6b30039650 0x7f6b3003bb10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b38006fd0 tx=0x7f6b38006e40 comp rx=0 tx=0).stop 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.140+0000 7f6b41600700 1 -- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 msgr2=0x7f6b4c1a3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.140+0000 7f6b41600700 1 --2- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c1a3240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009520 tx=0x7f6b3c011aa0 comp rx=0 tx=0).stop 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.141+0000 7f6b41600700 1 -- 172.21.15.106:0/1116457397 shutdown_connections 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.141+0000 7f6b41600700 1 --2- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6b30039650 0x7f6b3003bb10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.141+0000 7f6b41600700 1 --2- 172.21.15.106:0/1116457397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b4c07ee60 0x7f6b4c1a3240 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.141+0000 7f6b41600700 1 -- 172.21.15.106:0/1116457397 >> 172.21.15.106:0/1116457397 conn(0x7f6b4c1042a0 msgr2=0x7f6b4c106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.141+0000 7f6b41600700 1 -- 172.21.15.106:0/1116457397 shutdown_connections 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:40:59.141+0000 7f6b41600700 1 -- 172.21.15.106:0/1116457397 wait complete. 2024-04-28T12:40:59.786 INFO:teuthology.orchestra.run.smithi106.stdout:Creating initial admin user... 2024-04-28T12:41:00.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:59 smithi106 ceph-mon[28081]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:41:00.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:00.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:40:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:01.545 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$PhzbyNyMTmULFN5HyEvko.mbVpj8j8duVmlDqdiH/FRWymqcSlnqm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714308060, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-28T12:41:01.546 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.381+0000 7f026ea00700 1 Processor -- start 2024-04-28T12:41:01.546 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.381+0000 7f026ea00700 1 -- start start 2024-04-28T12:41:01.546 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.382+0000 7f026ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026807d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:01.547 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.382+0000 7f026ea00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f026807d810 con 0x7f026807ede0 2024-04-28T12:41:01.547 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.382+0000 7f0267e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026807d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:01.547 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.383+0000 7f0267e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026807d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53236/0 (socket says 172.21.15.106:53236) 2024-04-28T12:41:01.548 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.383+0000 7f0267e00700 1 -- 172.21.15.106:0/4082502050 learned_addr learned my addr 172.21.15.106:0/4082502050 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:01.548 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.383+0000 7f0267e00700 1 -- 172.21.15.106:0/4082502050 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026807d990 con 0x7f026807ede0 2024-04-28T12:41:01.548 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.383+0000 7f0267e00700 1 --2- 172.21.15.106:0/4082502050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026807d240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0258009ab0 tx=0x7f0258009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b6ad9815a4b8fa server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:01.548 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.384+0000 7f0266a00700 1 -- 172.21.15.106:0/4082502050 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0258004160 con 0x7f026807ede0 2024-04-28T12:41:01.549 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.384+0000 7f0266a00700 1 -- 172.21.15.106:0/4082502050 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f025800bc90 con 0x7f026807ede0 2024-04-28T12:41:01.549 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.384+0000 7f0266a00700 1 -- 172.21.15.106:0/4082502050 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0258003be0 con 0x7f026807ede0 2024-04-28T12:41:01.549 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.384+0000 7f026ea00700 1 -- 172.21.15.106:0/4082502050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 msgr2=0x7f026807d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:01.549 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.384+0000 7f026ea00700 1 --2- 172.21.15.106:0/4082502050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026807d240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0258009ab0 tx=0x7f0258009dc0 comp rx=0 tx=0).stop 2024-04-28T12:41:01.549 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.384+0000 7f026ea00700 1 -- 172.21.15.106:0/4082502050 shutdown_connections 2024-04-28T12:41:01.550 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.384+0000 7f026ea00700 1 --2- 172.21.15.106:0/4082502050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026807d240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:01.550 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.384+0000 7f026ea00700 1 -- 172.21.15.106:0/4082502050 >> 172.21.15.106:0/4082502050 conn(0x7f02681042f0 msgr2=0x7f0268106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:01.550 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.385+0000 7f026ea00700 1 -- 172.21.15.106:0/4082502050 shutdown_connections 2024-04-28T12:41:01.550 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.385+0000 7f026ea00700 1 -- 172.21.15.106:0/4082502050 wait complete. 2024-04-28T12:41:01.551 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.386+0000 7f026ea00700 1 Processor -- start 2024-04-28T12:41:01.551 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.386+0000 7f026ea00700 1 -- start start 2024-04-28T12:41:01.551 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.386+0000 7f026ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:01.551 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.386+0000 7f026ea00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f026810f330 con 0x7f026807ede0 2024-04-28T12:41:01.553 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.386+0000 7f0267e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:01.553 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.386+0000 7f0267e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026819eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53242/0 (socket says 172.21.15.106:53242) 2024-04-28T12:41:01.553 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.386+0000 7f0267e00700 1 -- 172.21.15.106:0/1781793737 learned_addr learned my addr 172.21.15.106:0/1781793737 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.387+0000 7f0267e00700 1 -- 172.21.15.106:0/1781793737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026819f3e0 con 0x7f026807ede0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.387+0000 7f0267e00700 1 --2- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026819eea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02580037a0 tx=0x7f025800bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.387+0000 7f0264c00700 1 -- 172.21.15.106:0/1781793737 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02580041d0 con 0x7f026807ede0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.387+0000 7f0264c00700 1 -- 172.21.15.106:0/1781793737 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0258004370 con 0x7f026807ede0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.387+0000 7f0264c00700 1 -- 172.21.15.106:0/1781793737 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0258011420 con 0x7f026807ede0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.387+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026819f670 con 0x7f026807ede0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.387+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02681a2360 con 0x7f026807ede0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.389+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0268064a80 con 0x7f026807ede0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.389+0000 7f0264c00700 1 -- 172.21.15.106:0/1781793737 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f025801b440 con 0x7f026807ede0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.390+0000 7f0264c00700 1 --2- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f024c039600 0x7f024c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.390+0000 7f0264c00700 1 -- 172.21.15.106:0/1781793737 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f025804deb0 con 0x7f026807ede0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.390+0000 7f0267400700 1 --2- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f024c039600 0x7f024c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:01.554 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.391+0000 7f0267400700 1 --2- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f024c039600 0x7f024c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0250006fd0 tx=0x7f0250006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:01.555 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.395+0000 7f0264c00700 1 -- 172.21.15.106:0/1781793737 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f025801f030 con 0x7f026807ede0 2024-04-28T12:41:01.555 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.629+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f02681a25f0 con 0x7f024c039600 2024-04-28T12:41:01.555 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.861+0000 7f0264c00700 1 -- 172.21.15.106:0/1781793737 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f02681a25f0 con 0x7f024c039600 2024-04-28T12:41:01.555 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.863+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f024c039600 msgr2=0x7f024c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:01.555 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.863+0000 7f026ea00700 1 --2- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f024c039600 0x7f024c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0250006fd0 tx=0x7f0250006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:01.556 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.863+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 msgr2=0x7f026819eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:01.556 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.863+0000 7f026ea00700 1 --2- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026819eea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f02580037a0 tx=0x7f025800bac0 comp rx=0 tx=0).stop 2024-04-28T12:41:01.556 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.863+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 shutdown_connections 2024-04-28T12:41:01.556 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.863+0000 7f026ea00700 1 --2- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f024c039600 0x7f024c03bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:01.556 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.863+0000 7f026ea00700 1 --2- 172.21.15.106:0/1781793737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f026807ede0 0x7f026819eea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:01.556 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.863+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 >> 172.21.15.106:0/1781793737 conn(0x7f02681042f0 msgr2=0x7f0268104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:01.556 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.864+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 shutdown_connections 2024-04-28T12:41:01.557 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:00.864+0000 7f026ea00700 1 -- 172.21.15.106:0/1781793737 wait complete. 2024-04-28T12:41:01.557 INFO:teuthology.orchestra.run.smithi106.stdout:Fetching dashboard port number... 2024-04-28T12:41:01.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:01 smithi106 ceph-mon[28081]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:41:01.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:01.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:02.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:02 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1216633854' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-28T12:41:03.756 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:03 smithi106 ceph-mon[28081]: mgrmap e13: x(active, since 6s) 2024-04-28T12:41:04.052 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stdout 8443 2024-04-28T12:41:04.053 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.218+0000 7f80c2800700 1 Processor -- start 2024-04-28T12:41:04.053 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.218+0000 7f80c2800700 1 -- start start 2024-04-28T12:41:04.053 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.219+0000 7f80c2800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:04.053 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.219+0000 7f80c2800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc1096f0 con 0x7f80bc108d00 2024-04-28T12:41:04.053 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.219+0000 7f80bb400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:04.053 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.219+0000 7f80bb400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc109120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53254/0 (socket says 172.21.15.106:53254) 2024-04-28T12:41:04.054 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.219+0000 7f80bb400700 1 -- 172.21.15.106:0/537253933 learned_addr learned my addr 172.21.15.106:0/537253933 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:04.054 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.220+0000 7f80bb400700 1 -- 172.21.15.106:0/537253933 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc109f00 con 0x7f80bc108d00 2024-04-28T12:41:04.054 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.220+0000 7f80bb400700 1 --2- 172.21.15.106:0/537253933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc109120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0098a0 tx=0x7f80ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786220cc8327e457 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:04.054 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.220+0000 7f80ba000700 1 -- 172.21.15.106:0/537253933 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80ac004160 con 0x7f80bc108d00 2024-04-28T12:41:04.055 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.220+0000 7f80ba000700 1 -- 172.21.15.106:0/537253933 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80ac004300 con 0x7f80bc108d00 2024-04-28T12:41:04.055 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.220+0000 7f80ba000700 1 -- 172.21.15.106:0/537253933 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80ac011a20 con 0x7f80bc108d00 2024-04-28T12:41:04.055 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.221+0000 7f80c2800700 1 -- 172.21.15.106:0/537253933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 msgr2=0x7f80bc109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:04.056 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.221+0000 7f80c2800700 1 --2- 172.21.15.106:0/537253933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc109120 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0098a0 tx=0x7f80ac009bb0 comp rx=0 tx=0).stop 2024-04-28T12:41:04.056 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.221+0000 7f80c2800700 1 -- 172.21.15.106:0/537253933 shutdown_connections 2024-04-28T12:41:04.056 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.221+0000 7f80c2800700 1 --2- 172.21.15.106:0/537253933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc109120 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:04.056 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.221+0000 7f80c2800700 1 -- 172.21.15.106:0/537253933 >> 172.21.15.106:0/537253933 conn(0x7f80bc1042c0 msgr2=0x7f80bc1066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:04.057 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.221+0000 7f80c2800700 1 -- 172.21.15.106:0/537253933 shutdown_connections 2024-04-28T12:41:04.057 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.221+0000 7f80c2800700 1 -- 172.21.15.106:0/537253933 wait complete. 2024-04-28T12:41:04.057 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.222+0000 7f80c2800700 1 Processor -- start 2024-04-28T12:41:04.057 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.223+0000 7f80c2800700 1 -- start start 2024-04-28T12:41:04.058 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.223+0000 7f80c2800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:04.058 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.223+0000 7f80c2800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc10d2b0 con 0x7f80bc108d00 2024-04-28T12:41:04.058 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.223+0000 7f80bb400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:04.059 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.223+0000 7f80bb400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53266/0 (socket says 172.21.15.106:53266) 2024-04-28T12:41:04.060 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.223+0000 7f80bb400700 1 -- 172.21.15.106:0/1216633854 learned_addr learned my addr 172.21.15.106:0/1216633854 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:04.060 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.224+0000 7f80bb400700 1 -- 172.21.15.106:0/1216633854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc19d1b0 con 0x7f80bc108d00 2024-04-28T12:41:04.060 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.224+0000 7f80bb400700 1 --2- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc19cc70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f80ac000c00 tx=0x7f80ac011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:04.060 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.224+0000 7f80b3e00700 1 -- 172.21.15.106:0/1216633854 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80ac003aa0 con 0x7f80bc108d00 2024-04-28T12:41:04.060 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.224+0000 7f80b3e00700 1 -- 172.21.15.106:0/1216633854 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80ac01b8f0 con 0x7f80bc108d00 2024-04-28T12:41:04.060 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.224+0000 7f80b3e00700 1 -- 172.21.15.106:0/1216633854 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80ac023890 con 0x7f80bc108d00 2024-04-28T12:41:04.060 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.224+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc19d440 con 0x7f80bc108d00 2024-04-28T12:41:04.060 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.224+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc1a0130 con 0x7f80bc108d00 2024-04-28T12:41:04.060 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.226+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc064a80 con 0x7f80bc108d00 2024-04-28T12:41:04.061 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.226+0000 7f80b3e00700 1 -- 172.21.15.106:0/1216633854 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f80ac01b440 con 0x7f80bc108d00 2024-04-28T12:41:04.061 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.226+0000 7f80b3e00700 1 --2- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f80a0039660 0x7f80a003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:04.061 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.226+0000 7f80b3e00700 1 -- 172.21.15.106:0/1216633854 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f80ac04dfa0 con 0x7f80bc108d00 2024-04-28T12:41:04.061 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.226+0000 7f80baa00700 1 --2- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f80a0039660 0x7f80a003bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:04.062 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.227+0000 7f80baa00700 1 --2- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f80a0039660 0x7f80a003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80a8006fd0 tx=0x7f80a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:04.062 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.232+0000 7f80b3e00700 1 -- 172.21.15.106:0/1216633854 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f80ac01f030 con 0x7f80bc108d00 2024-04-28T12:41:04.062 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.460+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f80bc1a0300 con 0x7f80bc108d00 2024-04-28T12:41:04.062 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.461+0000 7f80b3e00700 1 -- 172.21.15.106:0/1216633854 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f80ac00edf0 con 0x7f80bc108d00 2024-04-28T12:41:04.063 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.465+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f80a0039660 msgr2=0x7f80a003bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:04.063 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.465+0000 7f80c2800700 1 --2- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f80a0039660 0x7f80a003bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f80a8006fd0 tx=0x7f80a8006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:04.063 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.465+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 msgr2=0x7f80bc19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:04.063 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.465+0000 7f80c2800700 1 --2- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc19cc70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f80ac000c00 tx=0x7f80ac011aa0 comp rx=0 tx=0).stop 2024-04-28T12:41:04.063 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.465+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 shutdown_connections 2024-04-28T12:41:04.063 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.465+0000 7f80c2800700 1 --2- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f80a0039660 0x7f80a003bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:04.063 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.465+0000 7f80c2800700 1 --2- 172.21.15.106:0/1216633854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80bc108d00 0x7f80bc19cc70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:04.063 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.465+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 >> 172.21.15.106:0/1216633854 conn(0x7f80bc1042c0 msgr2=0x7f80bc104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:04.064 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.466+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 shutdown_connections 2024-04-28T12:41:04.064 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:02.466+0000 7f80c2800700 1 -- 172.21.15.106:0/1216633854 wait complete. 2024-04-28T12:41:04.101 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-28T12:41:04.101 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout disabled 2024-04-28T12:41:04.126 INFO:teuthology.orchestra.run.smithi106.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-28T12:41:04.126 INFO:teuthology.orchestra.run.smithi106.stdout:systemctl: stdout inactive 2024-04-28T12:41:04.126 INFO:teuthology.orchestra.run.smithi106.stdout:firewalld.service is not enabled 2024-04-28T12:41:04.126 INFO:teuthology.orchestra.run.smithi106.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-28T12:41:04.130 INFO:teuthology.orchestra.run.smithi106.stdout:Ceph Dashboard is now available at: 2024-04-28T12:41:04.130 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:04.130 INFO:teuthology.orchestra.run.smithi106.stdout: URL: https://smithi106.front.sepia.ceph.com:8443/ 2024-04-28T12:41:04.130 INFO:teuthology.orchestra.run.smithi106.stdout: User: admin 2024-04-28T12:41:04.130 INFO:teuthology.orchestra.run.smithi106.stdout: Password: yi8sdtu5by 2024-04-28T12:41:04.130 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:04.130 INFO:teuthology.orchestra.run.smithi106.stdout:Saving cluster configuration to /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config directory 2024-04-28T12:41:04.132 INFO:teuthology.orchestra.run.smithi106.stdout:Enabling autotune for osd_memory_target 2024-04-28T12:41:05.608 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.676+0000 7f280ae00700 1 Processor -- start 2024-04-28T12:41:05.608 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.676+0000 7f280ae00700 1 -- start start 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.676+0000 7f280ae00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f2804109180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.676+0000 7f280ae00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2804109750 con 0x7f2804108d60 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.677+0000 7f2803e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f2804109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.677+0000 7f2803e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f2804109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53276/0 (socket says 172.21.15.106:53276) 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.677+0000 7f2803e00700 1 -- 172.21.15.106:0/3487268762 learned_addr learned my addr 172.21.15.106:0/3487268762 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.677+0000 7f2803e00700 1 -- 172.21.15.106:0/3487268762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2804109f60 con 0x7f2804108d60 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.678+0000 7f2803e00700 1 --2- 172.21.15.106:0/3487268762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f2804109180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098a0 tx=0x7f27f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404fa74759a0f816 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.678+0000 7f2803400700 1 -- 172.21.15.106:0/3487268762 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f4004160 con 0x7f2804108d60 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.678+0000 7f2803400700 1 -- 172.21.15.106:0/3487268762 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f4004300 con 0x7f2804108d60 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.678+0000 7f2803400700 1 -- 172.21.15.106:0/3487268762 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f4011a20 con 0x7f2804108d60 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.679+0000 7f280ae00700 1 -- 172.21.15.106:0/3487268762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 msgr2=0x7f2804109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.679+0000 7f280ae00700 1 --2- 172.21.15.106:0/3487268762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f2804109180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f27f40098a0 tx=0x7f27f4009bb0 comp rx=0 tx=0).stop 2024-04-28T12:41:05.609 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.679+0000 7f280ae00700 1 -- 172.21.15.106:0/3487268762 shutdown_connections 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.679+0000 7f280ae00700 1 --2- 172.21.15.106:0/3487268762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f2804109180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.679+0000 7f280ae00700 1 -- 172.21.15.106:0/3487268762 >> 172.21.15.106:0/3487268762 conn(0x7f2804104320 msgr2=0x7f2804106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.679+0000 7f280ae00700 1 -- 172.21.15.106:0/3487268762 shutdown_connections 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.679+0000 7f280ae00700 1 -- 172.21.15.106:0/3487268762 wait complete. 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.680+0000 7f280ae00700 1 Processor -- start 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.680+0000 7f280ae00700 1 -- start start 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.680+0000 7f280ae00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f280419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.680+0000 7f280ae00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f280410d310 con 0x7f2804108d60 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.681+0000 7f2803e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f280419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.681+0000 7f2803e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f280419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53290/0 (socket says 172.21.15.106:53290) 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.681+0000 7f2803e00700 1 -- 172.21.15.106:0/515308271 learned_addr learned my addr 172.21.15.106:0/515308271 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:05.610 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.681+0000 7f2803e00700 1 -- 172.21.15.106:0/515308271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280419f3a0 con 0x7f2804108d60 2024-04-28T12:41:05.611 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.681+0000 7f2803e00700 1 --2- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f280419ee60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27f4000c00 tx=0x7f27f4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:05.611 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.682+0000 7f2801600700 1 -- 172.21.15.106:0/515308271 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f4003aa0 con 0x7f2804108d60 2024-04-28T12:41:05.611 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.682+0000 7f2801600700 1 -- 172.21.15.106:0/515308271 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f401b8f0 con 0x7f2804108d60 2024-04-28T12:41:05.613 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.682+0000 7f2801600700 1 -- 172.21.15.106:0/515308271 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27f4023890 con 0x7f2804108d60 2024-04-28T12:41:05.614 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.682+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280419f630 con 0x7f2804108d60 2024-04-28T12:41:05.614 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.682+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28041a2320 con 0x7f2804108d60 2024-04-28T12:41:05.614 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.683+0000 7f2801600700 1 -- 172.21.15.106:0/515308271 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f27f401b440 con 0x7f2804108d60 2024-04-28T12:41:05.614 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.683+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280410d490 con 0x7f2804108d60 2024-04-28T12:41:05.614 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.683+0000 7f2801600700 1 --2- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f27e8039700 0x7f27e803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:05.614 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.684+0000 7f2801600700 1 -- 172.21.15.106:0/515308271 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27f404e0d0 con 0x7f2804108d60 2024-04-28T12:41:05.614 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.684+0000 7f27fb400700 1 --2- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f27e8039700 0x7f27e803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.685+0000 7f27fb400700 1 --2- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f27e8039700 0x7f27e803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27ec006fd0 tx=0x7f27ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.689+0000 7f2801600700 1 -- 172.21.15.106:0/515308271 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f27f401f030 con 0x7f2804108d60 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.913+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f28041a2970 con 0x7f2804108d60 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.914+0000 7f2801600700 1 -- 172.21.15.106:0/515308271 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f27f40189c0 con 0x7f2804108d60 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.916+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f27e8039700 msgr2=0x7f27e803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.916+0000 7f280ae00700 1 --2- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f27e8039700 0x7f27e803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f27ec006fd0 tx=0x7f27ec006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.916+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 msgr2=0x7f280419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.916+0000 7f280ae00700 1 --2- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f280419ee60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27f4000c00 tx=0x7f27f4011aa0 comp rx=0 tx=0).stop 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.916+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 shutdown_connections 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.916+0000 7f280ae00700 1 --2- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f27e8039700 0x7f27e803bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.916+0000 7f280ae00700 1 --2- 172.21.15.106:0/515308271 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2804108d60 0x7f280419ee60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.916+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 >> 172.21.15.106:0/515308271 conn(0x7f2804104320 msgr2=0x7f2804104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.917+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 shutdown_connections 2024-04-28T12:41:05.615 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:04.917+0000 7f280ae00700 1 -- 172.21.15.106:0/515308271 wait complete. 2024-04-28T12:41:07.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.222+0000 7f1504000700 1 Processor -- start 2024-04-28T12:41:07.252 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.222+0000 7f1504000700 1 -- start start 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.223+0000 7f1504000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc07d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.223+0000 7f1504000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc07d870 con 0x7f14fc07ee40 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.224+0000 7f1501600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc07d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.224+0000 7f1501600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc07d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53304/0 (socket says 172.21.15.106:53304) 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.224+0000 7f1501600700 1 -- 172.21.15.106:0/638294791 learned_addr learned my addr 172.21.15.106:0/638294791 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.224+0000 7f1501600700 1 -- 172.21.15.106:0/638294791 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc07d9f0 con 0x7f14fc07ee40 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.224+0000 7f1501600700 1 --2- 172.21.15.106:0/638294791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc07d2a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009e80 tx=0x7f14f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac2665ed697f767 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.224+0000 7f14fbe00700 1 -- 172.21.15.106:0/638294791 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f0017070 con 0x7f14fc07ee40 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.225+0000 7f14fbe00700 1 -- 172.21.15.106:0/638294791 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f000b8c0 con 0x7f14fc07ee40 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.225+0000 7f14fbe00700 1 -- 172.21.15.106:0/638294791 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f0004490 con 0x7f14fc07ee40 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.225+0000 7f1504000700 1 -- 172.21.15.106:0/638294791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 msgr2=0x7f14fc07d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.225+0000 7f1504000700 1 --2- 172.21.15.106:0/638294791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc07d2a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009e80 tx=0x7f14f000b330 comp rx=0 tx=0).stop 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.225+0000 7f1504000700 1 -- 172.21.15.106:0/638294791 shutdown_connections 2024-04-28T12:41:07.253 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.225+0000 7f1504000700 1 --2- 172.21.15.106:0/638294791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc07d2a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.225+0000 7f1504000700 1 -- 172.21.15.106:0/638294791 >> 172.21.15.106:0/638294791 conn(0x7f14fc1042f0 msgr2=0x7f14fc106710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.226+0000 7f1504000700 1 -- 172.21.15.106:0/638294791 shutdown_connections 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.226+0000 7f1504000700 1 -- 172.21.15.106:0/638294791 wait complete. 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.226+0000 7f1504000700 1 Processor -- start 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.227+0000 7f1504000700 1 -- start start 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.227+0000 7f1504000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.227+0000 7f1504000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc10f2f0 con 0x7f14fc07ee40 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.227+0000 7f1501600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.227+0000 7f1501600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53306/0 (socket says 172.21.15.106:53306) 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.227+0000 7f1501600700 1 -- 172.21.15.106:0/2976621094 learned_addr learned my addr 172.21.15.106:0/2976621094 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.228+0000 7f1501600700 1 -- 172.21.15.106:0/2976621094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc19f360 con 0x7f14fc07ee40 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.228+0000 7f1501600700 1 --2- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc19ee20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f14f000bca0 tx=0x7f14f000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:07.254 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.228+0000 7f14fa000700 1 -- 172.21.15.106:0/2976621094 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f0020020 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.228+0000 7f14fa000700 1 -- 172.21.15.106:0/2976621094 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f0003b50 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.228+0000 7f14fa000700 1 -- 172.21.15.106:0/2976621094 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f0017040 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.228+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc19f5f0 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.228+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1a22e0 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.230+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc064a80 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.230+0000 7f14fa000700 1 -- 172.21.15.106:0/2976621094 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f14f0023020 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.230+0000 7f14fa000700 1 --2- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14e40396f0 0x7f14e403bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.230+0000 7f14fa000700 1 -- 172.21.15.106:0/2976621094 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14f000f050 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.230+0000 7f1500c00700 1 --2- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14e40396f0 0x7f14e403bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.231+0000 7f1500c00700 1 --2- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14e40396f0 0x7f14e403bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f14e8006fd0 tx=0x7f14e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.236+0000 7f14fa000700 1 -- 172.21.15.106:0/2976621094 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f14f002a020 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.573+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f14fc1a28b0 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.577+0000 7f14fa000700 1 -- 172.21.15.106:0/2976621094 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f14f002c300 con 0x7f14fc07ee40 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14e40396f0 msgr2=0x7f14e403bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:07.255 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 --2- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14e40396f0 0x7f14e403bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f14e8006fd0 tx=0x7f14e8006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 msgr2=0x7f14fc19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 --2- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc19ee20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f14f000bca0 tx=0x7f14f000bd80 comp rx=0 tx=0).stop 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 shutdown_connections 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 --2- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14e40396f0 0x7f14e403bbb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 --2- 172.21.15.106:0/2976621094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14fc07ee40 0x7f14fc19ee20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 >> 172.21.15.106:0/2976621094 conn(0x7f14fc1042f0 msgr2=0x7f14fc104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 shutdown_connections 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr 2024-04-28T12:41:06.580+0000 7f1504000700 1 -- 172.21.15.106:0/2976621094 wait complete. 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:07.256 INFO:teuthology.orchestra.run.smithi106.stdout:Or, if you are only running a single cluster on this host: 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout: ceph telemetry on 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout:For more information see: 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:07.257 INFO:teuthology.orchestra.run.smithi106.stdout:Bootstrap complete. 2024-04-28T12:41:07.328 INFO:tasks.cephadm:Fetching config... 2024-04-28T12:41:07.329 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:41:07.329 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-28T12:41:07.372 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-28T12:41:07.372 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:41:07.372 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-28T12:41:07.450 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-28T12:41:07.451 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:41:07.451 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-28T12:41:07.561 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-28T12:41:07.561 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:41:07.561 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-28T12:41:07.641 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-28T12:41:07.641 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKlBDNJmjY05MfSLHae6C/aVTAyDhX09M6wCgVXgTAhJPi5e2y/c41gywq+YGdsPUoHb9AramtQFFrnecYfyNz53eGt9FvraqKn7YKBXKx/9fl8i2D6hbH0faDeyu7uAwe8zQ/6mzLsC+wTqNTNr6r87P34I26WYObqOCDPPHZ4DV20hhLa8uVK9VWHlh+wBJr5uY1WQGEB3ZmZCRtmkqu8MB+qakkk1cVRxwvcSqfQCd1DZYDq4fS+DGBo+7Dr2/VcVwezJcv1SK6J3SUzHI0k5ncpKRLwBclyPt9yPnc35IzNluBb9cEwv3bjlmdchhfTU9COQ4dOzjfPUp4O7lAR6g80YZy30F3Z8GwtT+ayJkR3T1Bg/1NzWF9ohwkFusX33+vADtYqCVYLxZ4QWiqlkHCB9fhWNTrczc/83upSqihY3j2MbvFlExVmQou4YSZsvcYeQtoBFGe4xhN368V/dD5cFveOCdKzHUru025fqFQWMcDtGxrCksVkCH+OUU= ceph-e5e35dfa-055b-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-28T12:41:07.772 INFO:teuthology.orchestra.run.smithi106.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKlBDNJmjY05MfSLHae6C/aVTAyDhX09M6wCgVXgTAhJPi5e2y/c41gywq+YGdsPUoHb9AramtQFFrnecYfyNz53eGt9FvraqKn7YKBXKx/9fl8i2D6hbH0faDeyu7uAwe8zQ/6mzLsC+wTqNTNr6r87P34I26WYObqOCDPPHZ4DV20hhLa8uVK9VWHlh+wBJr5uY1WQGEB3ZmZCRtmkqu8MB+qakkk1cVRxwvcSqfQCd1DZYDq4fS+DGBo+7Dr2/VcVwezJcv1SK6J3SUzHI0k5ncpKRLwBclyPt9yPnc35IzNluBb9cEwv3bjlmdchhfTU9COQ4dOzjfPUp4O7lAR6g80YZy30F3Z8GwtT+ayJkR3T1Bg/1NzWF9ohwkFusX33+vADtYqCVYLxZ4QWiqlkHCB9fhWNTrczc/83upSqihY3j2MbvFlExVmQou4YSZsvcYeQtoBFGe4xhN368V/dD5cFveOCdKzHUru025fqFQWMcDtGxrCksVkCH+OUU= ceph-e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:41:07.799 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:07 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2976621094' entity='client.admin' 2024-04-28T12:41:07.806 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKlBDNJmjY05MfSLHae6C/aVTAyDhX09M6wCgVXgTAhJPi5e2y/c41gywq+YGdsPUoHb9AramtQFFrnecYfyNz53eGt9FvraqKn7YKBXKx/9fl8i2D6hbH0faDeyu7uAwe8zQ/6mzLsC+wTqNTNr6r87P34I26WYObqOCDPPHZ4DV20hhLa8uVK9VWHlh+wBJr5uY1WQGEB3ZmZCRtmkqu8MB+qakkk1cVRxwvcSqfQCd1DZYDq4fS+DGBo+7Dr2/VcVwezJcv1SK6J3SUzHI0k5ncpKRLwBclyPt9yPnc35IzNluBb9cEwv3bjlmdchhfTU9COQ4dOzjfPUp4O7lAR6g80YZy30F3Z8GwtT+ayJkR3T1Bg/1NzWF9ohwkFusX33+vADtYqCVYLxZ4QWiqlkHCB9fhWNTrczc/83upSqihY3j2MbvFlExVmQou4YSZsvcYeQtoBFGe4xhN368V/dD5cFveOCdKzHUru025fqFQWMcDtGxrCksVkCH+OUU= ceph-e5e35dfa-055b-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-28T12:41:07.847 INFO:teuthology.orchestra.run.smithi129.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKlBDNJmjY05MfSLHae6C/aVTAyDhX09M6wCgVXgTAhJPi5e2y/c41gywq+YGdsPUoHb9AramtQFFrnecYfyNz53eGt9FvraqKn7YKBXKx/9fl8i2D6hbH0faDeyu7uAwe8zQ/6mzLsC+wTqNTNr6r87P34I26WYObqOCDPPHZ4DV20hhLa8uVK9VWHlh+wBJr5uY1WQGEB3ZmZCRtmkqu8MB+qakkk1cVRxwvcSqfQCd1DZYDq4fS+DGBo+7Dr2/VcVwezJcv1SK6J3SUzHI0k5ncpKRLwBclyPt9yPnc35IzNluBb9cEwv3bjlmdchhfTU9COQ4dOzjfPUp4O7lAR6g80YZy30F3Z8GwtT+ayJkR3T1Bg/1NzWF9ohwkFusX33+vADtYqCVYLxZ4QWiqlkHCB9fhWNTrczc/83upSqihY3j2MbvFlExVmQou4YSZsvcYeQtoBFGe4xhN368V/dD5cFveOCdKzHUru025fqFQWMcDtGxrCksVkCH+OUU= ceph-e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:41:07.859 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKlBDNJmjY05MfSLHae6C/aVTAyDhX09M6wCgVXgTAhJPi5e2y/c41gywq+YGdsPUoHb9AramtQFFrnecYfyNz53eGt9FvraqKn7YKBXKx/9fl8i2D6hbH0faDeyu7uAwe8zQ/6mzLsC+wTqNTNr6r87P34I26WYObqOCDPPHZ4DV20hhLa8uVK9VWHlh+wBJr5uY1WQGEB3ZmZCRtmkqu8MB+qakkk1cVRxwvcSqfQCd1DZYDq4fS+DGBo+7Dr2/VcVwezJcv1SK6J3SUzHI0k5ncpKRLwBclyPt9yPnc35IzNluBb9cEwv3bjlmdchhfTU9COQ4dOzjfPUp4O7lAR6g80YZy30F3Z8GwtT+ayJkR3T1Bg/1NzWF9ohwkFusX33+vADtYqCVYLxZ4QWiqlkHCB9fhWNTrczc/83upSqihY3j2MbvFlExVmQou4YSZsvcYeQtoBFGe4xhN368V/dD5cFveOCdKzHUru025fqFQWMcDtGxrCksVkCH+OUU= ceph-e5e35dfa-055b-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-28T12:41:07.901 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKlBDNJmjY05MfSLHae6C/aVTAyDhX09M6wCgVXgTAhJPi5e2y/c41gywq+YGdsPUoHb9AramtQFFrnecYfyNz53eGt9FvraqKn7YKBXKx/9fl8i2D6hbH0faDeyu7uAwe8zQ/6mzLsC+wTqNTNr6r87P34I26WYObqOCDPPHZ4DV20hhLa8uVK9VWHlh+wBJr5uY1WQGEB3ZmZCRtmkqu8MB+qakkk1cVRxwvcSqfQCd1DZYDq4fS+DGBo+7Dr2/VcVwezJcv1SK6J3SUzHI0k5ncpKRLwBclyPt9yPnc35IzNluBb9cEwv3bjlmdchhfTU9COQ4dOzjfPUp4O7lAR6g80YZy30F3Z8GwtT+ayJkR3T1Bg/1NzWF9ohwkFusX33+vADtYqCVYLxZ4QWiqlkHCB9fhWNTrczc/83upSqihY3j2MbvFlExVmQou4YSZsvcYeQtoBFGe4xhN368V/dD5cFveOCdKzHUru025fqFQWMcDtGxrCksVkCH+OUU= ceph-e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:41:07.913 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-28T12:41:08.306 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:41:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.162+0000 7f96e7400700 1 -- 172.21.15.106:0/3901893201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 msgr2=0x7f96e01015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.162+0000 7f96e7400700 1 --2- 172.21.15.106:0/3901893201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 0x7f96e01015b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009910 tx=0x7f96d4009c20 comp rx=0 tx=0).stop 2024-04-28T12:41:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.162+0000 7f96e7400700 1 -- 172.21.15.106:0/3901893201 shutdown_connections 2024-04-28T12:41:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.162+0000 7f96e7400700 1 --2- 172.21.15.106:0/3901893201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 0x7f96e01015b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.162+0000 7f96e7400700 1 -- 172.21.15.106:0/3901893201 >> 172.21.15.106:0/3901893201 conn(0x7f96e00fcaf0 msgr2=0x7f96e00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.162+0000 7f96e7400700 1 -- 172.21.15.106:0/3901893201 shutdown_connections 2024-04-28T12:41:11.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.162+0000 7f96e7400700 1 -- 172.21.15.106:0/3901893201 wait complete. 2024-04-28T12:41:11.160 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:11.160 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:11.160 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:41:11.160 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:11.160 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:41:11.160 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:11.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.163+0000 7f96e7400700 1 Processor -- start 2024-04-28T12:41:11.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.164+0000 7f96e7400700 1 -- start start 2024-04-28T12:41:11.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.164+0000 7f96e7400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 0x7f96e019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:11.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.164+0000 7f96e7400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96e0108d90 con 0x7f96e0101190 2024-04-28T12:41:11.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.164+0000 7f96e4a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 0x7f96e019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:11.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.164+0000 7f96e4a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 0x7f96e019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45890/0 (socket says 172.21.15.106:45890) 2024-04-28T12:41:11.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.164+0000 7f96e4a00700 1 -- 172.21.15.106:0/2244976363 learned_addr learned my addr 172.21.15.106:0/2244976363 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:11.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.165+0000 7f96e4a00700 1 -- 172.21.15.106:0/2244976363 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e019abd0 con 0x7f96e0101190 2024-04-28T12:41:11.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.165+0000 7f96e4a00700 1 --2- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 0x7f96e019a690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f96d4005800 tx=0x7f96d4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:11.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.165+0000 7f96dd600700 1 -- 172.21.15.106:0/2244976363 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d4027070 con 0x7f96e0101190 2024-04-28T12:41:11.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.165+0000 7f96dd600700 1 -- 172.21.15.106:0/2244976363 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96d400be10 con 0x7f96e0101190 2024-04-28T12:41:11.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.165+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e019aec0 con 0x7f96e0101190 2024-04-28T12:41:11.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.165+0000 7f96dd600700 1 -- 172.21.15.106:0/2244976363 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d400f460 con 0x7f96e0101190 2024-04-28T12:41:11.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.166+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e019db50 con 0x7f96e0101190 2024-04-28T12:41:11.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.167+0000 7f96dd600700 1 -- 172.21.15.106:0/2244976363 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f96d402a070 con 0x7f96e0101190 2024-04-28T12:41:11.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.167+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e0109510 con 0x7f96e0101190 2024-04-28T12:41:11.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.168+0000 7f96dd600700 1 --2- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96c80396b0 0x7f96c803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:11.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.168+0000 7f96dd600700 1 -- 172.21.15.106:0/2244976363 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f96d401e030 con 0x7f96e0101190 2024-04-28T12:41:11.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.168+0000 7f96dfe00700 1 --2- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96c80396b0 0x7f96c803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:11.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.169+0000 7f96dfe00700 1 --2- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96c80396b0 0x7f96c803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f96cc006fd0 tx=0x7f96cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:11.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.174+0000 7f96dd600700 1 -- 172.21.15.106:0/2244976363 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f96d404fe20 con 0x7f96e0101190 2024-04-28T12:41:11.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.394+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f96e002e680 con 0x7f96e0101190 2024-04-28T12:41:11.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.397+0000 7f96dd600700 1 -- 172.21.15.106:0/2244976363 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f96d4031020 con 0x7f96e0101190 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96c80396b0 msgr2=0x7f96c803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 --2- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96c80396b0 0x7f96c803bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f96cc006fd0 tx=0x7f96cc006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 msgr2=0x7f96e019a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 --2- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 0x7f96e019a690 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f96d4005800 tx=0x7f96d4009f60 comp rx=0 tx=0).stop 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 shutdown_connections 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 --2- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96c80396b0 0x7f96c803bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 --2- 172.21.15.106:0/2244976363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96e0101190 0x7f96e019a690 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 >> 172.21.15.106:0/2244976363 conn(0x7f96e00fcaf0 msgr2=0x7f96e00feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 shutdown_connections 2024-04-28T12:41:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:11.401+0000 7f96e7400700 1 -- 172.21.15.106:0/2244976363 wait complete. 2024-04-28T12:41:12.170 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-28T12:41:12.171 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-28T12:41:12.818 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:12 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2244976363' entity='client.admin' 2024-04-28T12:41:12.819 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:41:12.819 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:12.906 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:41:14.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.864+0000 7feb74200700 1 -- 172.21.15.106:0/1802276049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 msgr2=0x7feb6c109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:14.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.864+0000 7feb74200700 1 --2- 172.21.15.106:0/1802276049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 0x7feb6c109a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feb60009910 tx=0x7feb60009c20 comp rx=0 tx=0).stop 2024-04-28T12:41:14.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.864+0000 7feb74200700 1 -- 172.21.15.106:0/1802276049 shutdown_connections 2024-04-28T12:41:14.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.864+0000 7feb74200700 1 --2- 172.21.15.106:0/1802276049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 0x7feb6c109a50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:14.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.864+0000 7feb74200700 1 -- 172.21.15.106:0/1802276049 >> 172.21.15.106:0/1802276049 conn(0x7feb6c078710 msgr2=0x7feb6c07ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:14.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.864+0000 7feb74200700 1 -- 172.21.15.106:0/1802276049 shutdown_connections 2024-04-28T12:41:14.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.864+0000 7feb74200700 1 -- 172.21.15.106:0/1802276049 wait complete. 2024-04-28T12:41:14.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.865+0000 7feb74200700 1 Processor -- start 2024-04-28T12:41:14.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.865+0000 7feb74200700 1 -- start start 2024-04-28T12:41:14.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.866+0000 7feb74200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 0x7feb6c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:14.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.866+0000 7feb74200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10f350 con 0x7feb6c109630 2024-04-28T12:41:14.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.866+0000 7feb71800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 0x7feb6c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:14.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.866+0000 7feb71800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 0x7feb6c19eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45894/0 (socket says 172.21.15.106:45894) 2024-04-28T12:41:14.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.866+0000 7feb71800700 1 -- 172.21.15.106:0/2529361749 learned_addr learned my addr 172.21.15.106:0/2529361749 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:14.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.866+0000 7feb71800700 1 -- 172.21.15.106:0/2529361749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c19eff0 con 0x7feb6c109630 2024-04-28T12:41:14.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.867+0000 7feb71800700 1 --2- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 0x7feb6c19eab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb60005190 tx=0x7feb60009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:14.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.867+0000 7feb6a000700 1 -- 172.21.15.106:0/2529361749 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb60027070 con 0x7feb6c109630 2024-04-28T12:41:14.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.867+0000 7feb6a000700 1 -- 172.21.15.106:0/2529361749 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb60022470 con 0x7feb6c109630 2024-04-28T12:41:14.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.867+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c19f280 con 0x7feb6c109630 2024-04-28T12:41:14.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.867+0000 7feb6a000700 1 -- 172.21.15.106:0/2529361749 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb6000f5f0 con 0x7feb6c109630 2024-04-28T12:41:14.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.867+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a1f70 con 0x7feb6c109630 2024-04-28T12:41:14.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.868+0000 7feb6a000700 1 -- 172.21.15.106:0/2529361749 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7feb6002a070 con 0x7feb6c109630 2024-04-28T12:41:14.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.868+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c10fad0 con 0x7feb6c109630 2024-04-28T12:41:14.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.869+0000 7feb6a000700 1 --2- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7feb540396b0 0x7feb5403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:14.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.869+0000 7feb6a000700 1 -- 172.21.15.106:0/2529361749 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feb6001e030 con 0x7feb6c109630 2024-04-28T12:41:14.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.869+0000 7feb70e00700 1 --2- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7feb540396b0 0x7feb5403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:14.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.870+0000 7feb70e00700 1 --2- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7feb540396b0 0x7feb5403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feb58006fd0 tx=0x7feb58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:14.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:14.874+0000 7feb6a000700 1 -- 172.21.15.106:0/2529361749 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7feb60013070 con 0x7feb6c109630 2024-04-28T12:41:15.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.101+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7feb6c02e5a0 con 0x7feb540396b0 2024-04-28T12:41:15.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.109+0000 7feb6a000700 1 -- 172.21.15.106:0/2529361749 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feb6c02e5a0 con 0x7feb540396b0 2024-04-28T12:41:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7feb540396b0 msgr2=0x7feb5403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 --2- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7feb540396b0 0x7feb5403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feb58006fd0 tx=0x7feb58006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 msgr2=0x7feb6c19eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:15.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 --2- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 0x7feb6c19eab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb60005190 tx=0x7feb60009ea0 comp rx=0 tx=0).stop 2024-04-28T12:41:15.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 shutdown_connections 2024-04-28T12:41:15.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 --2- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7feb540396b0 0x7feb5403bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:15.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 --2- 172.21.15.106:0/2529361749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb6c109630 0x7feb6c19eab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:15.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 >> 172.21.15.106:0/2529361749 conn(0x7feb6c078710 msgr2=0x7feb6c079220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:15.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 shutdown_connections 2024-04-28T12:41:15.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:15.112+0000 7feb74200700 1 -- 172.21.15.106:0/2529361749 wait complete. 2024-04-28T12:41:15.881 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi129 2024-04-28T12:41:15.908 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:41:15.908 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.conf 2024-04-28T12:41:15.926 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:41:15.926 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:41:15.983 INFO:tasks.cephadm:Adding host smithi129 to orchestrator... 2024-04-28T12:41:15.983 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch host add smithi129 2024-04-28T12:41:16.394 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:41:16.538 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:16 smithi106 ceph-mon[28081]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:41:16.538 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:16.538 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:41:16.538 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:16.538 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:41:17.566 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:17 smithi106 ceph-mon[28081]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-28T12:41:17.566 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:17 smithi106 ceph-mon[28081]: Updating smithi106:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:41:17.566 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:17 smithi106 ceph-mon[28081]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:18.220 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:18 smithi106 ceph-mon[28081]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:41:18.220 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:18 smithi106 ceph-mon[28081]: Updating smithi106:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-28T12:41:18.220 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:18.220 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:18.220 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:18.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.389+0000 7f9bc5000700 1 -- 172.21.15.106:0/2472402340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 msgr2=0x7f9bc010f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:18.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.389+0000 7f9bc5000700 1 --2- 172.21.15.106:0/2472402340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 0x7f9bc010f720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009910 tx=0x7f9bb0009c20 comp rx=0 tx=0).stop 2024-04-28T12:41:18.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.389+0000 7f9bc5000700 1 -- 172.21.15.106:0/2472402340 shutdown_connections 2024-04-28T12:41:18.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.389+0000 7f9bc5000700 1 --2- 172.21.15.106:0/2472402340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 0x7f9bc010f720 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:18.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.389+0000 7f9bc5000700 1 -- 172.21.15.106:0/2472402340 >> 172.21.15.106:0/2472402340 conn(0x7f9bc0078a20 msgr2=0x7f9bc007ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:18.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.390+0000 7f9bc5000700 1 -- 172.21.15.106:0/2472402340 shutdown_connections 2024-04-28T12:41:18.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.390+0000 7f9bc5000700 1 -- 172.21.15.106:0/2472402340 wait complete. 2024-04-28T12:41:18.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.391+0000 7f9bc5000700 1 Processor -- start 2024-04-28T12:41:18.388 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.391+0000 7f9bc5000700 1 -- start start 2024-04-28T12:41:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.391+0000 7f9bc5000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 0x7f9bc01a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.391+0000 7f9bc5000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc0113740 con 0x7f9bc010f340 2024-04-28T12:41:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.391+0000 7f9bbf400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 0x7f9bc01a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.392+0000 7f9bbf400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 0x7f9bc01a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45910/0 (socket says 172.21.15.106:45910) 2024-04-28T12:41:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.392+0000 7f9bbf400700 1 -- 172.21.15.106:0/3102336755 learned_addr learned my addr 172.21.15.106:0/3102336755 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.392+0000 7f9bbf400700 1 -- 172.21.15.106:0/3102336755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc01a4de0 con 0x7f9bc010f340 2024-04-28T12:41:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.392+0000 7f9bbf400700 1 --2- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 0x7f9bc01a48a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006010 tx=0x7f9bb0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:18.389 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.392+0000 7f9bb7e00700 1 -- 172.21.15.106:0/3102336755 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb0026070 con 0x7f9bc010f340 2024-04-28T12:41:18.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.392+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc01a50d0 con 0x7f9bc010f340 2024-04-28T12:41:18.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.392+0000 7f9bb7e00700 1 -- 172.21.15.106:0/3102336755 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb000bba0 con 0x7f9bc010f340 2024-04-28T12:41:18.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.392+0000 7f9bb7e00700 1 -- 172.21.15.106:0/3102336755 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb0003930 con 0x7f9bc010f340 2024-04-28T12:41:18.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.393+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc019ea60 con 0x7f9bc010f340 2024-04-28T12:41:18.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.394+0000 7f9bb7e00700 1 -- 172.21.15.106:0/3102336755 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f9bb0029070 con 0x7f9bc010f340 2024-04-28T12:41:18.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.394+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc010a980 con 0x7f9bc010f340 2024-04-28T12:41:18.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.394+0000 7f9bb7e00700 1 --2- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ba40396b0 0x7f9ba403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:18.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.394+0000 7f9bb7e00700 1 -- 172.21.15.106:0/3102336755 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9bb001e070 con 0x7f9bc010f340 2024-04-28T12:41:18.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.395+0000 7f9bbea00700 1 --2- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ba40396b0 0x7f9ba403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:18.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.395+0000 7f9bbea00700 1 --2- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ba40396b0 0x7f9ba403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9bac006fd0 tx=0x7f9bac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:18.396 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.400+0000 7f9bb7e00700 1 -- 172.21.15.106:0/3102336755 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9bb0013070 con 0x7f9bc010f340 2024-04-28T12:41:18.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:18.631+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}) -- 0x7f9bc01a00a0 con 0x7f9ba40396b0 2024-04-28T12:41:19.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:19 smithi106 ceph-mon[28081]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:19.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:19 smithi106 ceph-mon[28081]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:41:20.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:20 smithi106 ceph-mon[28081]: Deploying cephadm binary to smithi129 2024-04-28T12:41:20.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:20 smithi106 ceph-mon[28081]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:23.308 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:22 smithi106 ceph-mon[28081]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:23.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.378+0000 7f9bb7e00700 1 -- 172.21.15.106:0/3102336755 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9bc01a00a0 con 0x7f9ba40396b0 2024-04-28T12:41:23.374 INFO:teuthology.orchestra.run.smithi106.stdout:Added host 'smithi129' with addr '172.21.15.129' 2024-04-28T12:41:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.381+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ba40396b0 msgr2=0x7f9ba403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.381+0000 7f9bc5000700 1 --2- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ba40396b0 0x7f9ba403bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9bac006fd0 tx=0x7f9bac006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.381+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 msgr2=0x7f9bc01a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.381+0000 7f9bc5000700 1 --2- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 0x7f9bc01a48a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0006010 tx=0x7f9bb0005780 comp rx=0 tx=0).stop 2024-04-28T12:41:23.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.381+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 shutdown_connections 2024-04-28T12:41:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.381+0000 7f9bc5000700 1 --2- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ba40396b0 0x7f9ba403bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.381+0000 7f9bc5000700 1 --2- 172.21.15.106:0/3102336755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9bc010f340 0x7f9bc01a48a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.381+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 >> 172.21.15.106:0/3102336755 conn(0x7f9bc0078a20 msgr2=0x7f9bc007a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.382+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 shutdown_connections 2024-04-28T12:41:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:23.382+0000 7f9bc5000700 1 -- 172.21.15.106:0/3102336755 wait complete. 2024-04-28T12:41:24.097 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-28T12:41:24.520 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:41:24.667 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:24.667 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:24 smithi106 ceph-mon[28081]: Added host smithi129 2024-04-28T12:41:24.668 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:41:24.668 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:24 smithi106 ceph-mon[28081]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:24.668 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:25.644 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:25 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.461+0000 7fa487200700 1 -- 172.21.15.106:0/3472712466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 msgr2=0x7fa4881032a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.461+0000 7fa487200700 1 --2- 172.21.15.106:0/3472712466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 0x7fa4881032a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa478009910 tx=0x7fa478009c20 comp rx=0 tx=0).stop 2024-04-28T12:41:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.461+0000 7fa487200700 1 -- 172.21.15.106:0/3472712466 shutdown_connections 2024-04-28T12:41:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.461+0000 7fa487200700 1 --2- 172.21.15.106:0/3472712466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 0x7fa4881032a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.461+0000 7fa487200700 1 -- 172.21.15.106:0/3472712466 >> 172.21.15.106:0/3472712466 conn(0x7fa4880fe7b0 msgr2=0x7fa488100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:26.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.461+0000 7fa487200700 1 -- 172.21.15.106:0/3472712466 shutdown_connections 2024-04-28T12:41:26.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.461+0000 7fa487200700 1 -- 172.21.15.106:0/3472712466 wait complete. 2024-04-28T12:41:26.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.462+0000 7fa487200700 1 Processor -- start 2024-04-28T12:41:26.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.462+0000 7fa487200700 1 -- start start 2024-04-28T12:41:26.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.463+0000 7fa487200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 0x7fa4881a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:26.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.463+0000 7fa487200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa48810d140 con 0x7fa488102ec0 2024-04-28T12:41:26.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.463+0000 7fa485e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 0x7fa4881a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:26.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.463+0000 7fa485e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 0x7fa4881a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49854/0 (socket says 172.21.15.106:49854) 2024-04-28T12:41:26.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.463+0000 7fa485e00700 1 -- 172.21.15.106:0/2098482416 learned_addr learned my addr 172.21.15.106:0/2098482416 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:26.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.464+0000 7fa485e00700 1 -- 172.21.15.106:0/2098482416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4881a0b50 con 0x7fa488102ec0 2024-04-28T12:41:26.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.464+0000 7fa485e00700 1 --2- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 0x7fa4881a0610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4780056d0 tx=0x7fa4780057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:26.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.465+0000 7fa47ea00700 1 -- 172.21.15.106:0/2098482416 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa478026070 con 0x7fa488102ec0 2024-04-28T12:41:26.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.465+0000 7fa47ea00700 1 -- 172.21.15.106:0/2098482416 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47800bba0 con 0x7fa488102ec0 2024-04-28T12:41:26.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.465+0000 7fa47ea00700 1 -- 172.21.15.106:0/2098482416 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa478003920 con 0x7fa488102ec0 2024-04-28T12:41:26.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.465+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4881a0e40 con 0x7fa488102ec0 2024-04-28T12:41:26.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.466+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48819a8b0 con 0x7fa488102ec0 2024-04-28T12:41:26.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.467+0000 7fa47ea00700 1 -- 172.21.15.106:0/2098482416 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fa478029070 con 0x7fa488102ec0 2024-04-28T12:41:26.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.467+0000 7fa47ea00700 1 --2- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa46c0396b0 0x7fa46c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:26.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.467+0000 7fa47ea00700 1 -- 172.21.15.106:0/2098482416 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa47801e070 con 0x7fa488102ec0 2024-04-28T12:41:26.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.467+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4881043a0 con 0x7fa488102ec0 2024-04-28T12:41:26.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.468+0000 7fa485400700 1 --2- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa46c0396b0 0x7fa46c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:26.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.468+0000 7fa485400700 1 --2- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa46c0396b0 0x7fa46c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa474006fd0 tx=0x7fa474006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.475+0000 7fa47ea00700 1 -- 172.21.15.106:0/2098482416 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa478013070 con 0x7fa488102ec0 2024-04-28T12:41:26.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:26 smithi106 ceph-mon[28081]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:26.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:26 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:26.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.706+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa48819bc10 con 0x7fa46c0396b0 2024-04-28T12:41:26.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.709+0000 7fa47ea00700 1 -- 172.21.15.106:0/2098482416 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fa48819bc10 con 0x7fa46c0396b0 2024-04-28T12:41:26.705 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:26.705 INFO:teuthology.orchestra.run.smithi106.stdout:[{"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}, {"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}] 2024-04-28T12:41:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.711+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa46c0396b0 msgr2=0x7fa46c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:26.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.711+0000 7fa487200700 1 --2- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa46c0396b0 0x7fa46c03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa474006fd0 tx=0x7fa474006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.711+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 msgr2=0x7fa4881a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.711+0000 7fa487200700 1 --2- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 0x7fa4881a0610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4780056d0 tx=0x7fa4780057b0 comp rx=0 tx=0).stop 2024-04-28T12:41:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.712+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 shutdown_connections 2024-04-28T12:41:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.712+0000 7fa487200700 1 --2- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa46c0396b0 0x7fa46c03bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.712+0000 7fa487200700 1 --2- 172.21.15.106:0/2098482416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa488102ec0 0x7fa4881a0610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.712+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 >> 172.21.15.106:0/2098482416 conn(0x7fa4880fe7b0 msgr2=0x7fa488075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.712+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 shutdown_connections 2024-04-28T12:41:26.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:26.712+0000 7fa487200700 1 -- 172.21.15.106:0/2098482416 wait complete. 2024-04-28T12:41:27.449 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-04-28T12:41:27.449 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:41:27.449 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-04-28T12:41:27.468 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:41:27.468 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:41:27.526 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-04-28T12:41:27.526 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch host add smithi171 2024-04-28T12:41:27.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:27 smithi106 ceph-mon[28081]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:41:27.966 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:41:28.779 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:28 smithi106 ceph-mon[28081]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.831+0000 7f2d37e00700 1 -- 172.21.15.106:0/1713780988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 msgr2=0x7f2d3810f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.831+0000 7f2d37e00700 1 --2- 172.21.15.106:0/1713780988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 0x7f2d3810f720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009910 tx=0x7f2d28009c20 comp rx=0 tx=0).stop 2024-04-28T12:41:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.831+0000 7f2d37e00700 1 -- 172.21.15.106:0/1713780988 shutdown_connections 2024-04-28T12:41:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.831+0000 7f2d37e00700 1 --2- 172.21.15.106:0/1713780988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 0x7f2d3810f720 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.831+0000 7f2d37e00700 1 -- 172.21.15.106:0/1713780988 >> 172.21.15.106:0/1713780988 conn(0x7f2d38078a20 msgr2=0x7f2d3807ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.832+0000 7f2d37e00700 1 -- 172.21.15.106:0/1713780988 shutdown_connections 2024-04-28T12:41:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.832+0000 7f2d37e00700 1 -- 172.21.15.106:0/1713780988 wait complete. 2024-04-28T12:41:29.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.833+0000 7f2d37e00700 1 Processor -- start 2024-04-28T12:41:29.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.833+0000 7f2d37e00700 1 -- start start 2024-04-28T12:41:29.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.833+0000 7f2d37e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 0x7f2d381a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:29.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.833+0000 7f2d37e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38113740 con 0x7f2d3810f340 2024-04-28T12:41:29.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.834+0000 7f2d36a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 0x7f2d381a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:29.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.834+0000 7f2d36a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 0x7f2d381a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51924/0 (socket says 172.21.15.106:51924) 2024-04-28T12:41:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.834+0000 7f2d36a00700 1 -- 172.21.15.106:0/2738863151 learned_addr learned my addr 172.21.15.106:0/2738863151 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.834+0000 7f2d36a00700 1 -- 172.21.15.106:0/2738863151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d381a4de0 con 0x7f2d3810f340 2024-04-28T12:41:29.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.834+0000 7f2d36a00700 1 --2- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 0x7f2d381a48a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d28006010 tx=0x7f2d28005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:29.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.835+0000 7f2d2f400700 1 -- 172.21.15.106:0/2738863151 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d28026070 con 0x7f2d3810f340 2024-04-28T12:41:29.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.835+0000 7f2d2f400700 1 -- 172.21.15.106:0/2738863151 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d2800bba0 con 0x7f2d3810f340 2024-04-28T12:41:29.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.835+0000 7f2d2f400700 1 -- 172.21.15.106:0/2738863151 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d28003930 con 0x7f2d3810f340 2024-04-28T12:41:29.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.835+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d381a50d0 con 0x7f2d3810f340 2024-04-28T12:41:29.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.835+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3819ea60 con 0x7f2d3810f340 2024-04-28T12:41:29.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.836+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3810a980 con 0x7f2d3810f340 2024-04-28T12:41:29.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.836+0000 7f2d2f400700 1 -- 172.21.15.106:0/2738863151 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f2d28029070 con 0x7f2d3810f340 2024-04-28T12:41:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.837+0000 7f2d2f400700 1 --2- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d1c0396b0 0x7f2d1c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.837+0000 7f2d2f400700 1 -- 172.21.15.106:0/2738863151 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2d2801e070 con 0x7f2d3810f340 2024-04-28T12:41:29.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.837+0000 7f2d36000700 1 --2- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d1c0396b0 0x7f2d1c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:29.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.838+0000 7f2d36000700 1 --2- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d1c0396b0 0x7f2d1c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2d24006fd0 tx=0x7f2d24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:29.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:29.842+0000 7f2d2f400700 1 -- 172.21.15.106:0/2738863151 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2d28013070 con 0x7f2d3810f340 2024-04-28T12:41:30.073 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:30.076+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}) -- 0x7f2d381a00a0 con 0x7f2d1c0396b0 2024-04-28T12:41:30.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:30.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:30.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:30.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:30.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:41:30.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:30.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:41:30.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-28T12:41:30.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: Updating smithi129:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:41:30.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:30.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:41:30.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:30 smithi106 ceph-mon[28081]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:41:32.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:31 smithi106 ceph-mon[28081]: Updating smithi129:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-28T12:41:32.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:31 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:32.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:31 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:32.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:31 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:32.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:31 smithi106 ceph-mon[28081]: Deploying cephadm binary to smithi171 2024-04-28T12:41:33.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:32 smithi106 ceph-mon[28081]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:34.810 INFO:teuthology.orchestra.run.smithi106.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-04-28T12:41:34.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.812+0000 7f2d2f400700 1 -- 172.21.15.106:0/2738863151 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2d381a00a0 con 0x7f2d1c0396b0 2024-04-28T12:41:34.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.816+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d1c0396b0 msgr2=0x7f2d1c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:34.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.816+0000 7f2d37e00700 1 --2- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d1c0396b0 0x7f2d1c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2d24006fd0 tx=0x7f2d24006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:34.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.816+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 msgr2=0x7f2d381a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:34.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.816+0000 7f2d37e00700 1 --2- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 0x7f2d381a48a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d28006010 tx=0x7f2d28005780 comp rx=0 tx=0).stop 2024-04-28T12:41:34.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.816+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 shutdown_connections 2024-04-28T12:41:34.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.816+0000 7f2d37e00700 1 --2- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d1c0396b0 0x7f2d1c03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:34.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.816+0000 7f2d37e00700 1 --2- 172.21.15.106:0/2738863151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d3810f340 0x7f2d381a48a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:34.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.816+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 >> 172.21.15.106:0/2738863151 conn(0x7f2d38078a20 msgr2=0x7f2d3807a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:34.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.817+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 shutdown_connections 2024-04-28T12:41:34.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:34.817+0000 7f2d37e00700 1 -- 172.21.15.106:0/2738863151 wait complete. 2024-04-28T12:41:35.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:34 smithi106 ceph-mon[28081]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:35.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:34 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:35.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:34 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:41:35.547 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-28T12:41:35.994 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:41:36.256 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:35 smithi106 ceph-mon[28081]: Added host smithi171 2024-04-28T12:41:36.256 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:36.257 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:37.194 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:36 smithi106 ceph-mon[28081]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:37.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.951+0000 7fd547400700 1 -- 172.21.15.106:0/1037841846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 msgr2=0x7fd540103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.951+0000 7fd547400700 1 --2- 172.21.15.106:0/1037841846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 0x7fd540103450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd530009910 tx=0x7fd530009c20 comp rx=0 tx=0).stop 2024-04-28T12:41:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.951+0000 7fd547400700 1 -- 172.21.15.106:0/1037841846 shutdown_connections 2024-04-28T12:41:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.951+0000 7fd547400700 1 --2- 172.21.15.106:0/1037841846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 0x7fd540103450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.951+0000 7fd547400700 1 -- 172.21.15.106:0/1037841846 >> 172.21.15.106:0/1037841846 conn(0x7fd5400fe940 msgr2=0x7fd540100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.951+0000 7fd547400700 1 -- 172.21.15.106:0/1037841846 shutdown_connections 2024-04-28T12:41:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.951+0000 7fd547400700 1 -- 172.21.15.106:0/1037841846 wait complete. 2024-04-28T12:41:37.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.952+0000 7fd547400700 1 Processor -- start 2024-04-28T12:41:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.952+0000 7fd547400700 1 -- start start 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.953+0000 7fd547400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 0x7fd540192850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.953+0000 7fd547400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd54010d2f0 con 0x7fd540103070 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.953+0000 7fd544a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 0x7fd540192850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.953+0000 7fd544a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 0x7fd540192850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51942/0 (socket says 172.21.15.106:51942) 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.953+0000 7fd544a00700 1 -- 172.21.15.106:0/4081692376 learned_addr learned my addr 172.21.15.106:0/4081692376 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.953+0000 7fd544a00700 1 -- 172.21.15.106:0/4081692376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5401967b0 con 0x7fd540103070 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.954+0000 7fd544a00700 1 --2- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 0x7fd540192850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd530006010 tx=0x7fd530005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.954+0000 7fd53e000700 1 -- 172.21.15.106:0/4081692376 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd530026070 con 0x7fd540103070 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.954+0000 7fd53e000700 1 -- 172.21.15.106:0/4081692376 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd53000bba0 con 0x7fd540103070 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.954+0000 7fd53e000700 1 -- 172.21.15.106:0/4081692376 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd530003910 con 0x7fd540103070 2024-04-28T12:41:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.954+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd540192d90 con 0x7fd540103070 2024-04-28T12:41:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.954+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd540193250 con 0x7fd540103070 2024-04-28T12:41:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.955+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd540104550 con 0x7fd540103070 2024-04-28T12:41:37.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.955+0000 7fd53e000700 1 -- 172.21.15.106:0/4081692376 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fd530029070 con 0x7fd540103070 2024-04-28T12:41:37.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.956+0000 7fd53e000700 1 --2- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd524039650 0x7fd52403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:37.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.956+0000 7fd53e000700 1 -- 172.21.15.106:0/4081692376 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd53001e070 con 0x7fd540103070 2024-04-28T12:41:37.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.956+0000 7fd537e00700 1 --2- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd524039650 0x7fd52403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:37.953 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.957+0000 7fd537e00700 1 --2- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd524039650 0x7fd52403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd528006fd0 tx=0x7fd528006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:37.962+0000 7fd53e000700 1 -- 172.21.15.106:0/4081692376 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd530013070 con 0x7fd540103070 2024-04-28T12:41:38.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.197+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd540196980 con 0x7fd524039650 2024-04-28T12:41:38.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.198+0000 7fd53e000700 1 -- 172.21.15.106:0/4081692376 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fd540196980 con 0x7fd524039650 2024-04-28T12:41:38.195 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:41:38.195 INFO:teuthology.orchestra.run.smithi106.stdout:[{"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}, {"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-04-28T12:41:38.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.201+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd524039650 msgr2=0x7fd52403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:38.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.201+0000 7fd547400700 1 --2- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd524039650 0x7fd52403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd528006fd0 tx=0x7fd528006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:38.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.201+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 msgr2=0x7fd540192850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:38.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.201+0000 7fd547400700 1 --2- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 0x7fd540192850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd530006010 tx=0x7fd530005780 comp rx=0 tx=0).stop 2024-04-28T12:41:38.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.201+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 shutdown_connections 2024-04-28T12:41:38.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.201+0000 7fd547400700 1 --2- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd524039650 0x7fd52403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:38.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.201+0000 7fd547400700 1 --2- 172.21.15.106:0/4081692376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd540103070 0x7fd540192850 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:38.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.201+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 >> 172.21.15.106:0/4081692376 conn(0x7fd5400fe940 msgr2=0x7fd540075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:38.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.201+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 shutdown_connections 2024-04-28T12:41:38.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:38.202+0000 7fd547400700 1 -- 172.21.15.106:0/4081692376 wait complete. 2024-04-28T12:41:38.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:38 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:38.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:38 smithi106 ceph-mon[28081]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:38.908 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-28T12:41:38.908 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-28T12:41:39.329 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:41:39.796 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:39 smithi106 ceph-mon[28081]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:41:40.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:40 smithi106 ceph-mon[28081]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:41.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.366+0000 7fd48be00700 1 -- 172.21.15.106:0/1033822006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 msgr2=0x7fd484103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:41.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.366+0000 7fd48be00700 1 --2- 172.21.15.106:0/1033822006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 0x7fd484103440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd478009910 tx=0x7fd478009c20 comp rx=0 tx=0).stop 2024-04-28T12:41:41.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.366+0000 7fd48be00700 1 -- 172.21.15.106:0/1033822006 shutdown_connections 2024-04-28T12:41:41.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.366+0000 7fd48be00700 1 --2- 172.21.15.106:0/1033822006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 0x7fd484103440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:41.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.366+0000 7fd48be00700 1 -- 172.21.15.106:0/1033822006 >> 172.21.15.106:0/1033822006 conn(0x7fd4840fe910 msgr2=0x7fd484100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:41.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.366+0000 7fd48be00700 1 -- 172.21.15.106:0/1033822006 shutdown_connections 2024-04-28T12:41:41.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.366+0000 7fd48be00700 1 -- 172.21.15.106:0/1033822006 wait complete. 2024-04-28T12:41:41.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.368+0000 7fd48be00700 1 Processor -- start 2024-04-28T12:41:41.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.368+0000 7fd48be00700 1 -- start start 2024-04-28T12:41:41.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.368+0000 7fd48be00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 0x7fd4841a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:41.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.368+0000 7fd48be00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd48410d2e0 con 0x7fd484103060 2024-04-28T12:41:41.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.368+0000 7fd489400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 0x7fd4841a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:41.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.368+0000 7fd489400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 0x7fd4841a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51748/0 (socket says 172.21.15.106:51748) 2024-04-28T12:41:41.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.368+0000 7fd489400700 1 -- 172.21.15.106:0/242829690 learned_addr learned my addr 172.21.15.106:0/242829690 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:41:41.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.369+0000 7fd489400700 1 -- 172.21.15.106:0/242829690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4841a0c30 con 0x7fd484103060 2024-04-28T12:41:41.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.369+0000 7fd489400700 1 --2- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 0x7fd4841a06f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd478006010 tx=0x7fd478005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:41.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.369+0000 7fd482000700 1 -- 172.21.15.106:0/242829690 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd478026070 con 0x7fd484103060 2024-04-28T12:41:41.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.369+0000 7fd482000700 1 -- 172.21.15.106:0/242829690 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47800bba0 con 0x7fd484103060 2024-04-28T12:41:41.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.369+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4841a0f20 con 0x7fd484103060 2024-04-28T12:41:41.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.369+0000 7fd482000700 1 -- 172.21.15.106:0/242829690 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd478003910 con 0x7fd484103060 2024-04-28T12:41:41.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.370+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48419a8b0 con 0x7fd484103060 2024-04-28T12:41:41.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.371+0000 7fd482000700 1 -- 172.21.15.106:0/242829690 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fd478029070 con 0x7fd484103060 2024-04-28T12:41:41.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.371+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd484104540 con 0x7fd484103060 2024-04-28T12:41:41.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.371+0000 7fd482000700 1 --2- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd46c039650 0x7fd46c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:41.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.371+0000 7fd482000700 1 -- 172.21.15.106:0/242829690 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd47801e070 con 0x7fd484103060 2024-04-28T12:41:41.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.371+0000 7fd488a00700 1 --2- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd46c039650 0x7fd46c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:41.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.372+0000 7fd488a00700 1 --2- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd46c039650 0x7fd46c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd470006fd0 tx=0x7fd470006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:41.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.377+0000 7fd482000700 1 -- 172.21.15.106:0/242829690 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd478013070 con 0x7fd484103060 2024-04-28T12:41:41.609 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:41.613+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd48419bde0 con 0x7fd484103060 2024-04-28T12:41:41.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:41.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:41.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:41.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:41.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:41:41.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:41.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:41:41.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:41 smithi106 ceph-mon[28081]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-28T12:41:41.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:41 smithi106 ceph-mon[28081]: Updating smithi171:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:41:42.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.566+0000 7fd482000700 1 -- 172.21.15.106:0/242829690 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fd47804cfe0 con 0x7fd484103060 2024-04-28T12:41:42.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd46c039650 msgr2=0x7fd46c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:42.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 --2- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd46c039650 0x7fd46c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd470006fd0 tx=0x7fd470006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:42.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 msgr2=0x7fd4841a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:42.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 --2- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 0x7fd4841a06f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd478006010 tx=0x7fd478005780 comp rx=0 tx=0).stop 2024-04-28T12:41:42.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 shutdown_connections 2024-04-28T12:41:42.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 --2- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd46c039650 0x7fd46c03bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:42.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 --2- 172.21.15.106:0/242829690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd484103060 0x7fd4841a06f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:42.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 >> 172.21.15.106:0/242829690 conn(0x7fd4840fe910 msgr2=0x7fd484075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:42.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 shutdown_connections 2024-04-28T12:41:42.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:41:42.570+0000 7fd48be00700 1 -- 172.21.15.106:0/242829690 wait complete. 2024-04-28T12:41:42.568 INFO:teuthology.orchestra.run.smithi106.stderr:adjusted tunables profile to default 2024-04-28T12:41:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:42 smithi106 ceph-mon[28081]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:41:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:42 smithi106 ceph-mon[28081]: Updating smithi171:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-28T12:41:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:42 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/242829690' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-28T12:41:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:42 smithi106 ceph-mon[28081]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:43.350 INFO:tasks.cephadm:Adding mon.a on smithi106 2024-04-28T12:41:43.351 INFO:tasks.cephadm:Adding mon.b on smithi129 2024-04-28T12:41:43.351 INFO:tasks.cephadm:Adding mon.c on smithi171 2024-04-28T12:41:43.351 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi106:172.21.15.106=a;smithi129:172.21.15.129=b;smithi171:172.21.15.171=c' 2024-04-28T12:41:43.592 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:41:43.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:43 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/242829690' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-28T12:41:43.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:43 smithi106 ceph-mon[28081]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:41:44.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:44 smithi106 ceph-mon[28081]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:45.069 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.073+0000 7f987709e700 1 -- 172.21.15.171:0/1626100847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 msgr2=0x7f9870102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:45.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.073+0000 7f987709e700 1 --2- 172.21.15.171:0/1626100847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 0x7f9870102360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9864009910 tx=0x7f9864009c20 comp rx=0 tx=0).stop 2024-04-28T12:41:45.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.073+0000 7f987709e700 1 -- 172.21.15.171:0/1626100847 shutdown_connections 2024-04-28T12:41:45.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.073+0000 7f987709e700 1 --2- 172.21.15.171:0/1626100847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 0x7f9870102360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:45.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.073+0000 7f987709e700 1 -- 172.21.15.171:0/1626100847 >> 172.21.15.171:0/1626100847 conn(0x7f98700fd810 msgr2=0x7f98700ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:45.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.073+0000 7f987709e700 1 -- 172.21.15.171:0/1626100847 shutdown_connections 2024-04-28T12:41:45.070 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.073+0000 7f987709e700 1 -- 172.21.15.171:0/1626100847 wait complete. 2024-04-28T12:41:45.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.074+0000 7f987709e700 1 Processor -- start 2024-04-28T12:41:45.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.074+0000 7f987709e700 1 -- start start 2024-04-28T12:41:45.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.074+0000 7f987709e700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 0x7f98701a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:45.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.074+0000 7f987709e700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f987010c200 con 0x7f9870101f80 2024-04-28T12:41:45.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.075+0000 7f9874e3a700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 0x7f98701a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:45.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.075+0000 7f9874e3a700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 0x7f98701a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.171:43364/0 (socket says 172.21.15.171:43364) 2024-04-28T12:41:45.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.075+0000 7f9874e3a700 1 -- 172.21.15.171:0/2474406830 learned_addr learned my addr 172.21.15.171:0/2474406830 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:41:45.071 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.075+0000 7f9874e3a700 1 -- 172.21.15.171:0/2474406830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98701a0c70 con 0x7f9870101f80 2024-04-28T12:41:45.072 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.076+0000 7f9874e3a700 1 --2- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 0x7f98701a0730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98640056d0 tx=0x7f98640057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:45.072 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.076+0000 7f986dffb700 1 -- 172.21.15.171:0/2474406830 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9864026070 con 0x7f9870101f80 2024-04-28T12:41:45.072 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.076+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98701a0f60 con 0x7f9870101f80 2024-04-28T12:41:45.072 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.076+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987019a8f0 con 0x7f9870101f80 2024-04-28T12:41:45.072 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.076+0000 7f986dffb700 1 -- 172.21.15.171:0/2474406830 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f986400bba0 con 0x7f9870101f80 2024-04-28T12:41:45.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.076+0000 7f986dffb700 1 -- 172.21.15.171:0/2474406830 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98640037f0 con 0x7f9870101f80 2024-04-28T12:41:45.073 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.077+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9854005330 con 0x7f9870101f80 2024-04-28T12:41:45.074 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.078+0000 7f986dffb700 1 -- 172.21.15.171:0/2474406830 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f9864029070 con 0x7f9870101f80 2024-04-28T12:41:45.074 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.078+0000 7f986dffb700 1 --2- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f98600396b0 0x7f986003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:45.074 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.078+0000 7f986dffb700 1 -- 172.21.15.171:0/2474406830 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f986401e070 con 0x7f9870101f80 2024-04-28T12:41:45.074 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.078+0000 7f986ffff700 1 --2- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f98600396b0 0x7f986003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:45.075 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.079+0000 7f986ffff700 1 --2- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f98600396b0 0x7f986003bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f985c006fd0 tx=0x7f985c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:45.088 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.092+0000 7f986dffb700 1 -- 172.21.15.171:0/2474406830 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9864013070 con 0x7f9870101f80 2024-04-28T12:41:45.300 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.303+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi106:172.21.15.106=a;smithi129:172.21.15.129=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}) -- 0x7f9854000c80 con 0x7f98600396b0 2024-04-28T12:41:45.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.319+0000 7f986dffb700 1 -- 172.21.15.171:0/2474406830 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9854000c80 con 0x7f98600396b0 2024-04-28T12:41:45.318 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mon update... 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f98600396b0 msgr2=0x7f986003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 --2- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f98600396b0 0x7f986003bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f985c006fd0 tx=0x7f985c006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 msgr2=0x7f98701a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 --2- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 0x7f98701a0730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98640056d0 tx=0x7f98640057b0 comp rx=0 tx=0).stop 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 shutdown_connections 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 --2- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f98600396b0 0x7f986003bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 --2- 172.21.15.171:0/2474406830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9870101f80 0x7f98701a0730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 >> 172.21.15.171:0/2474406830 conn(0x7f98700fd810 msgr2=0x7f987006b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 shutdown_connections 2024-04-28T12:41:45.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:45.321+0000 7f987709e700 1 -- 172.21.15.171:0/2474406830 wait complete. 2024-04-28T12:41:45.887 DEBUG:teuthology.orchestra.run.smithi129:mon.b> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.b.service 2024-04-28T12:41:45.890 DEBUG:teuthology.orchestra.run.smithi171:mon.c> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.c.service 2024-04-28T12:41:45.892 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-28T12:41:45.892 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-28T12:41:46.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi106:172.21.15.106=a;smithi129:172.21.15.129=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:41:46.742 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: Saving service mon spec with placement smithi106:172.21.15.106=a;smithi129:172.21.15.129=b;smithi171:172.21.15.171=c;count:3 2024-04-28T12:41:46.742 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:46.742 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:41:46.743 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:46.743 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:41:46.743 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:46.743 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:41:46.743 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:46.743 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: Deploying daemon mon.c on smithi171 2024-04-28T12:41:46.743 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:46 smithi106 ceph-mon[28081]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:48.035 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:41:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:48 smithi106 ceph-mon[28081]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:49.702 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.705+0000 7fe21a26d700 1 -- 172.21.15.171:0/2556103684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 msgr2=0x7fe214102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:49.702 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.705+0000 7fe21a26d700 1 --2- 172.21.15.171:0/2556103684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 0x7fe214102460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe204009910 tx=0x7fe204009c20 comp rx=0 tx=0).stop 2024-04-28T12:41:49.702 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.705+0000 7fe21a26d700 1 -- 172.21.15.171:0/2556103684 shutdown_connections 2024-04-28T12:41:49.703 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.705+0000 7fe21a26d700 1 --2- 172.21.15.171:0/2556103684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 0x7fe214102460 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:49.703 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.705+0000 7fe21a26d700 1 -- 172.21.15.171:0/2556103684 >> 172.21.15.171:0/2556103684 conn(0x7fe2140fd950 msgr2=0x7fe2140ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:49.703 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.705+0000 7fe21a26d700 1 -- 172.21.15.171:0/2556103684 shutdown_connections 2024-04-28T12:41:49.703 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.705+0000 7fe21a26d700 1 -- 172.21.15.171:0/2556103684 wait complete. 2024-04-28T12:41:49.703 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.706+0000 7fe21a26d700 1 Processor -- start 2024-04-28T12:41:49.703 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.706+0000 7fe21a26d700 1 -- start start 2024-04-28T12:41:49.703 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.706+0000 7fe21a26d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 0x7fe21419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:49.703 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.706+0000 7fe21a26d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe21410c300 con 0x7fe214102080 2024-04-28T12:41:49.704 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.707+0000 7fe213fff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 0x7fe21419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:49.704 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.707+0000 7fe213fff700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 0x7fe21419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.171:55328/0 (socket says 172.21.15.171:55328) 2024-04-28T12:41:49.704 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.707+0000 7fe213fff700 1 -- 172.21.15.171:0/2960631704 learned_addr learned my addr 172.21.15.171:0/2960631704 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:41:49.705 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.707+0000 7fe213fff700 1 -- 172.21.15.171:0/2960631704 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21419eb50 con 0x7fe214102080 2024-04-28T12:41:49.705 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.708+0000 7fe213fff700 1 --2- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 0x7fe21419e610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe204006010 tx=0x7fe204005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:49.705 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.708+0000 7fe2117fa700 1 -- 172.21.15.171:0/2960631704 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe204026070 con 0x7fe214102080 2024-04-28T12:41:49.705 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.708+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21419ee40 con 0x7fe214102080 2024-04-28T12:41:49.705 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.708+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe214198790 con 0x7fe214102080 2024-04-28T12:41:49.705 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.708+0000 7fe2117fa700 1 -- 172.21.15.171:0/2960631704 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe20400b6f0 con 0x7fe214102080 2024-04-28T12:41:49.705 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.708+0000 7fe2117fa700 1 -- 172.21.15.171:0/2960631704 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe204030060 con 0x7fe214102080 2024-04-28T12:41:49.706 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.709+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe200005330 con 0x7fe214102080 2024-04-28T12:41:49.706 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.709+0000 7fe2117fa700 1 -- 172.21.15.171:0/2960631704 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fe204029070 con 0x7fe214102080 2024-04-28T12:41:49.706 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.710+0000 7fe2117fa700 1 --2- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc039650 0x7fe1fc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:49.706 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.710+0000 7fe2117fa700 1 -- 172.21.15.171:0/2960631704 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe20401e070 con 0x7fe214102080 2024-04-28T12:41:49.706 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.710+0000 7fe2137fe700 1 --2- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc039650 0x7fe1fc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:49.707 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.711+0000 7fe2137fe700 1 --2- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc039650 0x7fe1fc03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe208006fd0 tx=0x7fe208006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:49.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:49.725+0000 7fe2117fa700 1 -- 172.21.15.171:0/2960631704 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe204013070 con 0x7fe214102080 2024-04-28T12:41:50.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.042+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe2000058f0 con 0x7fe214102080 2024-04-28T12:41:50.040 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.044+0000 7fe2117fa700 1 -- 172.21.15.171:0/2960631704 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fe20402ba90 con 0x7fe214102080 2024-04-28T12:41:50.040 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:41:50.040 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":1,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","modified":"2024-04-28T12:39:45.237228Z","created":"2024-04-28T12:39:45.237228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-28T12:41:50.042 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc039650 msgr2=0x7fe1fc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:50.042 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 --2- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc039650 0x7fe1fc03bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe208006fd0 tx=0x7fe208006e40 comp rx=0 tx=0).stop 2024-04-28T12:41:50.042 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 msgr2=0x7fe21419e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:50.042 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 --2- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 0x7fe21419e610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe204006010 tx=0x7fe204005780 comp rx=0 tx=0).stop 2024-04-28T12:41:50.042 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 shutdown_connections 2024-04-28T12:41:50.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 --2- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc039650 0x7fe1fc03bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:50.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 --2- 172.21.15.171:0/2960631704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214102080 0x7fe21419e610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:50.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 >> 172.21.15.171:0/2960631704 conn(0x7fe2140fd950 msgr2=0x7fe21406b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:50.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 shutdown_connections 2024-04-28T12:41:50.043 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:50.046+0000 7fe21a26d700 1 -- 172.21.15.171:0/2960631704 wait complete. 2024-04-28T12:41:50.043 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 1 2024-04-28T12:41:51.894 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-28T12:41:51.895 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-28T12:41:52.169 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:41:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:41:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: mon.a calling monitor election 2024-04-28T12:41:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: mon.c calling monitor election 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: monmap epoch 2 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: last_changed 2024-04-28T12:41:50.163795+0000 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: min_mon_release 19 (squid) 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: election_strategy: 1 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-28T12:41:55.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: fsmap 2024-04-28T12:41:55.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:41:55.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: mgrmap e13: x(active, since 59s) 2024-04-28T12:41:55.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: overall HEALTH_OK 2024-04-28T12:41:55.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:55.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:56.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.556+0000 7ff028408700 1 -- 172.21.15.171:0/2591457197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0201040d0 msgr2=0x7ff00c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:56.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.556+0000 7ff028408700 1 --2- 172.21.15.171:0/2591457197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0201040d0 0x7ff00c005610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff018009bf0 tx=0x7ff018009f00 comp rx=0 tx=0).stop 2024-04-28T12:41:56.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.556+0000 7ff028408700 1 -- 172.21.15.171:0/2591457197 shutdown_connections 2024-04-28T12:41:56.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.556+0000 7ff028408700 1 --2- 172.21.15.171:0/2591457197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0201040d0 0x7ff00c005610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:56.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.556+0000 7ff028408700 1 -- 172.21.15.171:0/2591457197 >> 172.21.15.171:0/2591457197 conn(0x7ff0200fd900 msgr2=0x7ff0200ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:56.554 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.556+0000 7ff028408700 1 -- 172.21.15.171:0/2591457197 shutdown_connections 2024-04-28T12:41:56.554 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.556+0000 7ff028408700 1 -- 172.21.15.171:0/2591457197 wait complete. 2024-04-28T12:41:56.554 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:41:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:56.556 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.557+0000 7ff028408700 1 Processor -- start 2024-04-28T12:41:56.556 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.557+0000 7ff028408700 1 -- start start 2024-04-28T12:41:56.556 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff028408700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0201040d0 0x7ff02019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff028408700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 0x7ff0201987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff028408700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff02010ae60 con 0x7ff0201040d0 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff028408700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff02010ace0 con 0x7ff02019eb60 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff0259a3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 0x7ff0201987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff0259a3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 0x7ff0201987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:48468/0 (socket says 172.21.15.171:48468) 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff0259a3700 1 -- 172.21.15.171:0/3683682183 learned_addr learned my addr 172.21.15.171:0/3683682183 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff0259a3700 1 -- 172.21.15.171:0/3683682183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 msgr2=0x7ff0201987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff0259a3700 1 -- 172.21.15.171:0/3683682183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 msgr2=0x7ff0201987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff0259a3700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 0x7ff0201987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff0259a3700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 0x7ff0201987a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.558+0000 7ff0261a4700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0201040d0 0x7ff02019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:56.557 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.559+0000 7ff0261a4700 1 -- 172.21.15.171:0/3683682183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 msgr2=0x7ff0201987a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:41:56.558 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.559+0000 7ff0261a4700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 0x7ff0201987a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:56.558 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.559+0000 7ff0261a4700 1 -- 172.21.15.171:0/3683682183 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02019f1a0 con 0x7ff0201040d0 2024-04-28T12:41:56.558 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.559+0000 7ff0261a4700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0201040d0 0x7ff02019e620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff018009bf0 tx=0x7ff0180044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:56.558 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.559+0000 7ff0177fe700 1 -- 172.21.15.171:0/3683682183 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff018017470 con 0x7ff0201040d0 2024-04-28T12:41:56.558 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.559+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff020198d40 con 0x7ff0201040d0 2024-04-28T12:41:56.558 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.560+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0201992a0 con 0x7ff0201040d0 2024-04-28T12:41:56.558 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.560+0000 7ff0177fe700 1 -- 172.21.15.171:0/3683682183 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff018024030 con 0x7ff0201040d0 2024-04-28T12:41:56.558 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.560+0000 7ff0177fe700 1 -- 172.21.15.171:0/3683682183 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff0180206b0 con 0x7ff0201040d0 2024-04-28T12:41:56.559 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.561+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff02006b680 con 0x7ff0201040d0 2024-04-28T12:41:56.559 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.561+0000 7ff0177fe700 1 -- 172.21.15.171:0/3683682183 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7ff01801e070 con 0x7ff0201040d0 2024-04-28T12:41:56.559 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.561+0000 7ff0177fe700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0040393f0 0x7ff00403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:41:56.559 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.561+0000 7ff0177fe700 1 -- 172.21.15.171:0/3683682183 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff01801c070 con 0x7ff0201040d0 2024-04-28T12:41:56.559 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.562+0000 7ff0259a3700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0040393f0 0x7ff00403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:41:56.559 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.563+0000 7ff0259a3700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0040393f0 0x7ff00403b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff010009180 tx=0x7ff010008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:41:56.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:56.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:41:56.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:56.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:56 smithi106 ceph-mon[28081]: Deploying daemon mon.b on smithi129 2024-04-28T12:41:56.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:56 smithi106 ceph-mon[28081]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:56.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:56.576 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.579+0000 7ff0177fe700 1 -- 172.21.15.171:0/3683682183 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff018031550 con 0x7ff0201040d0 2024-04-28T12:41:56.854 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.858+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff020199580 con 0x7ff0201040d0 2024-04-28T12:41:56.855 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.859+0000 7ff0177fe700 1 -- 172.21.15.171:0/3683682183 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7ff018014d80 con 0x7ff0201040d0 2024-04-28T12:41:56.855 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:41:56.855 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","modified":"2024-04-28T12:41:50.163795Z","created":"2024-04-28T12:39:45.237228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-28T12:41:56.857 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0040393f0 msgr2=0x7ff00403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:56.857 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0040393f0 0x7ff00403b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff010009180 tx=0x7ff010008040 comp rx=0 tx=0).stop 2024-04-28T12:41:56.857 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0201040d0 msgr2=0x7ff02019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:41:56.857 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0201040d0 0x7ff02019e620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff018009bf0 tx=0x7ff0180044b0 comp rx=0 tx=0).stop 2024-04-28T12:41:56.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 shutdown_connections 2024-04-28T12:41:56.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0201040d0 0x7ff02019e620 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:56.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0040393f0 0x7ff00403b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:56.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 --2- 172.21.15.171:0/3683682183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff02019eb60 0x7ff0201987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:41:56.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 >> 172.21.15.171:0/3683682183 conn(0x7ff0200fd900 msgr2=0x7ff020104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:41:56.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 shutdown_connections 2024-04-28T12:41:56.858 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:41:56.861+0000 7ff028408700 1 -- 172.21.15.171:0/3683682183 wait complete. 2024-04-28T12:41:56.858 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-04-28T12:41:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:41:57 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/3683682183' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-28T12:41:57.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:57 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/3683682183' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-28T12:41:57.575 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:41:57 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:41:57.164+0000 7f2b92000700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-28T12:41:58.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:41:58 smithi106 ceph-mon[28081]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:58.612 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-28T12:41:58.612 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-28T12:41:58.639 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:41:58 smithi171 ceph-mon[63127]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:58.855 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:41:59.270 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_background_jobs: 2 2024-04-28T12:41:59.270 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_background_compactions: -1 2024-04-28T12:41:59.270 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_subcompactions: 1 2024-04-28T12:41:59.270 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-28T12:41:59.270 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-28T12:41:59.270 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-28T12:41:59.271 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_total_wal_size: 0 2024-04-28T12:41:59.271 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-28T12:41:59.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-28T12:41:59.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-28T12:41:59.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-28T12:41:59.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_open_files: -1 2024-04-28T12:41:59.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bytes_per_sync: 0 2024-04-28T12:41:59.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-28T12:41:59.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_readahead_size: 0 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_background_flushes: -1 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Compression algorithms supported: 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: kZSTD supported: 0 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: kXpressCompression supported: 0 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: kLZ4HCCompression supported: 1 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: kZlibCompression supported: 1 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: kSnappyCompression supported: 1 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: kLZ4Compression supported: 1 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: kBZip2Compression supported: 0 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-28T12:41:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.merge_operator: 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_filter: None 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_filter_factory: None 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.sst_partitioner_factory: None 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d51b2fff40) 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout: cache_index_and_filter_blocks: 1 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout: pin_top_level_index_and_filter: 1 2024-04-28T12:41:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout: index_type: 0 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: data_block_index_type: 0 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: index_shortening: 1 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: checksum: 4 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: no_block_cache: 0 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache: 0x55d51b3031f0 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache_name: BinnedLRUCache 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache_options: 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: capacity : 536870912 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: num_shard_bits : 4 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: strict_capacity_limit : 0 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: high_pri_pool_ratio: 0.000 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_cache_compressed: (nil) 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: persistent_cache: (nil) 2024-04-28T12:41:59.767 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_size: 4096 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_size_deviation: 10 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_restart_interval: 16 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: index_block_restart_interval: 1 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: metadata_block_size: 4096 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: partition_filters: 0 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: use_delta_encoding: 1 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: filter_policy: bloomfilter 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: whole_key_filtering: 1 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: verify_compression: 0 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: read_amp_bytes_per_bit: 0 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: format_version: 5 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: enable_index_compression: 1 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: block_align: 0 2024-04-28T12:41:59.768 INFO:journalctl@ceph.mon.b.smithi129.stdout: max_auto_readahead_size: 262144 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout: prepopulate_block_cache: 0 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout: initial_auto_readahead_size: 8192 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout: num_file_reads_for_auto_readahead: 2 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.write_buffer_size: 33554432 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_write_buffer_number: 2 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression: NoCompression 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression: Disabled 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.prefix_extractor: nullptr 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.num_levels: 7 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-28T12:41:59.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression_opts.level: 32767 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression_opts.strategy: 0 2024-04-28T12:41:59.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-28T12:41:59.772 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-28T12:41:59.772 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-28T12:41:59.772 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-28T12:41:59.772 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression_opts.enabled: false 2024-04-28T12:41:59.772 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-28T12:41:59.772 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-28T12:41:59.772 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-28T12:41:59.772 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-28T12:41:59.772 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.target_file_size_base: 67108864 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-28T12:41:59.773 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.arena_block_size: 1048576 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.disable_auto_compactions: 0 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-28T12:41:59.774 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.inplace_update_support: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.bloom_locality: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.max_successive_merges: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.paranoid_file_checks: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.force_consistency_checks: 1 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.report_bg_io_stats: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.ttl: 2592000 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-28T12:41:59.775 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.enable_blob_files: false 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.min_blob_size: 0 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.blob_file_size: 268435456 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.blob_file_starting_level: 0 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f8f49a58-4747-4974-8520-9ef98648a05c 2024-04-28T12:41:59.776 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714308119273439, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714308119274024, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714308119, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f8f49a58-4747-4974-8520-9ef98648a05c", "db_session_id": "FZDBH0CZRXMV2G1FJ4AE", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714308119274129, "job": 1, "event": "recovery_finished"} 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d51b420000 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: DB pointer 0x55d51b40c000 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** DB Stats ** 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-28T12:41:59.777 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** Compaction Stats [default] ** 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** Compaction Stats [default] ** 2024-04-28T12:41:59.778 INFO:journalctl@ceph.mon.b.smithi129.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: Cumulative compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: Interval compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-28T12:41:59.779 INFO:journalctl@ceph.mon.b.smithi129.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: Block cache BinnedLRUCache@0x55d51b3031f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: using public_addr v2:172.21.15.129:0/0 -> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: starting mon.b rank -1 at public addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] at bind addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(???) e0 preinit fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).mds e1 new map 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: e1 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: legacy client fscid: -1 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: 2024-04-28T12:41:59.780 INFO:journalctl@ceph.mon.b.smithi129.stdout: No filesystems configured 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Added host smithi129 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.781 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Updating smithi129:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:41:59.782 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Updating smithi129:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Deploying cephadm binary to smithi171 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Added host smithi171 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.783 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Updating smithi171:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Updating smithi171:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-28T12:41:59.784 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/242829690' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/242829690' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi106:172.21.15.106=a;smithi129:172.21.15.129=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Saving service mon spec with placement smithi106:172.21.15.106=a;smithi129:172.21.15.129=b;smithi171:172.21.15.171=c;count:3 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.785 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Deploying daemon mon.c on smithi171 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.a calling monitor election 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.c calling monitor election 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.786 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: monmap epoch 2 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: last_changed 2024-04-28T12:41:50.163795+0000 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: min_mon_release 19 (squid) 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: election_strategy: 1 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: fsmap 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mgrmap e13: x(active, since 59s) 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: overall HEALTH_OK 2024-04-28T12:41:59.787 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: Deploying daemon mon.b on smithi129 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/3683682183' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 ceph-mon[62418]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 bash[62399]: ff947229bf24b5d74c586529eb5fd666b0cd9b996178fbc9b484ce46ac6854e6 2024-04-28T12:41:59.788 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:41:59 smithi129 systemd[1]: Started Ceph mon.b for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:42:04.720 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.723+0000 7f5239a94700 1 -- 172.21.15.171:0/2560541284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 msgr2=0x7f5220007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:04.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.723+0000 7f5239a94700 1 --2- 172.21.15.171:0/2560541284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 0x7f5220007c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5234079550 tx=0x7f522400fe10 comp rx=0 tx=0).stop 2024-04-28T12:42:04.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.723+0000 7f5239a94700 1 -- 172.21.15.171:0/2560541284 shutdown_connections 2024-04-28T12:42:04.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.723+0000 7f5239a94700 1 --2- 172.21.15.171:0/2560541284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 0x7f5220007c60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:04.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.723+0000 7f5239a94700 1 -- 172.21.15.171:0/2560541284 >> 172.21.15.171:0/2560541284 conn(0x7f5234078240 msgr2=0x7f5234078650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:42:04.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.723+0000 7f5239a94700 1 -- 172.21.15.171:0/2560541284 shutdown_connections 2024-04-28T12:42:04.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.723+0000 7f5239a94700 1 -- 172.21.15.171:0/2560541284 wait complete. 2024-04-28T12:42:04.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.724+0000 7f5239a94700 1 Processor -- start 2024-04-28T12:42:04.721 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.724+0000 7f5239a94700 1 -- start start 2024-04-28T12:42:04.721 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: mon.c calling monitor election 2024-04-28T12:42:04.721 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: mon.a calling monitor election 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-28T12:42:04.722 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: monmap epoch 3 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: last_changed 2024-04-28T12:41:59.303534+0000 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: min_mon_release 19 (squid) 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: election_strategy: 1 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: fsmap 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: mgrmap e13: x(active, since 68s) 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: overall HEALTH_OK 2024-04-28T12:42:04.723 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:04.724 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:04.727 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.724+0000 7f5239a94700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 0x7f523419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:04.727 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.724+0000 7f5239a94700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f523419cb70 0x7f5234196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:04.727 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.724+0000 7f5239a94700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5234196c40 0x7f52341970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:04.727 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.724+0000 7f5239a94700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5234108d50 con 0x7f52200057d0 2024-04-28T12:42:04.727 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.724+0000 7f5239a94700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52341024e0 con 0x7f5234196c40 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.724+0000 7f5239a94700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5234108ed0 con 0x7f523419cb70 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f52337fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 0x7f523419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f5233fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5234196c40 0x7f52341970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f52337fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 0x7f523419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.171:56312/0 (socket says 172.21.15.171:56312) 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f52337fe700 1 -- 172.21.15.171:0/1152514452 learned_addr learned my addr 172.21.15.171:0/1152514452 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f522bfff700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f523419cb70 0x7f5234196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f5233fff700 1 -- 172.21.15.171:0/1152514452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5234196c40 msgr2=0x7f52341970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f5233fff700 1 -- 172.21.15.171:0/1152514452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5234196c40 msgr2=0x7f52341970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f5233fff700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5234196c40 0x7f52341970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f5233fff700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5234196c40 0x7f52341970f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f522bfff700 1 -- 172.21.15.171:0/1152514452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f523419cb70 msgr2=0x7f5234196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f522bfff700 1 -- 172.21.15.171:0/1152514452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f523419cb70 msgr2=0x7f5234196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-28T12:42:04.728 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f522bfff700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f523419cb70 0x7f5234196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f522bfff700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f523419cb70 0x7f5234196700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f52337fe700 1 -- 172.21.15.171:0/1152514452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f523419cb70 msgr2=0x7f5234196700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f52337fe700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f523419cb70 0x7f5234196700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f52337fe700 1 -- 172.21.15.171:0/1152514452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5234196c40 msgr2=0x7f52341970f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f52337fe700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5234196c40 0x7f52341970f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.725+0000 7f52337fe700 1 -- 172.21.15.171:0/1152514452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52341a33e0 con 0x7f52200057d0 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.726+0000 7f52337fe700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 0x7f523419c630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5224000c00 tx=0x7f5224013f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.726+0000 7f52317fa700 1 -- 172.21.15.171:0/1152514452 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224022040 con 0x7f52200057d0 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.726+0000 7f52317fa700 1 -- 172.21.15.171:0/1152514452 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f522402d070 con 0x7f52200057d0 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.726+0000 7f52317fa700 1 -- 172.21.15.171:0/1152514452 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522401db30 con 0x7f52200057d0 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.726+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52341a3610 con 0x7f52200057d0 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.726+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52341a3a40 con 0x7f52200057d0 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.727+0000 7f52317fa700 1 -- 172.21.15.171:0/1152514452 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f5224003680 con 0x7f52200057d0 2024-04-28T12:42:04.729 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.727+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5234103750 con 0x7f52200057d0 2024-04-28T12:42:04.730 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.728+0000 7f52317fa700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5214039ae0 0x7f521403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:04.730 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.728+0000 7f52317fa700 1 -- 172.21.15.171:0/1152514452 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f522404f490 con 0x7f52200057d0 2024-04-28T12:42:04.730 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.728+0000 7f522bfff700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5214039ae0 0x7f521403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:04.730 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.729+0000 7f522bfff700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5214039ae0 0x7f521403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f521c009850 tx=0x7f521c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:04.741 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:04.745+0000 7f52317fa700 1 -- 172.21.15.171:0/1152514452 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5224003970 con 0x7f52200057d0 2024-04-28T12:42:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: mon.c calling monitor election 2024-04-28T12:42:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:42:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: mon.a calling monitor election 2024-04-28T12:42:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: monmap epoch 3 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: last_changed 2024-04-28T12:41:59.303534+0000 2024-04-28T12:42:04.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: min_mon_release 19 (squid) 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: election_strategy: 1 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: fsmap 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: mgrmap e13: x(active, since 68s) 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: overall HEALTH_OK 2024-04-28T12:42:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:04.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:05.034 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.038+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5234064a80 con 0x7f52200057d0 2024-04-28T12:42:05.035 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.039+0000 7f52317fa700 1 -- 172.21.15.171:0/1152514452 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f522401d400 con 0x7f52200057d0 2024-04-28T12:42:05.035 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-04-28T12:42:05.035 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":3,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","modified":"2024-04-28T12:41:59.303534Z","created":"2024-04-28T12:39:45.237228Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5214039ae0 msgr2=0x7f521403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5214039ae0 0x7f521403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f521c009850 tx=0x7f521c008040 comp rx=0 tx=0).stop 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 msgr2=0x7f523419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 0x7f523419c630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5224000c00 tx=0x7f5224013f80 comp rx=0 tx=0).stop 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 shutdown_connections 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5234196c40 0x7f52341970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52200057d0 0x7f523419c630 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5214039ae0 0x7f521403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 --2- 172.21.15.171:0/1152514452 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f523419cb70 0x7f5234196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 >> 172.21.15.171:0/1152514452 conn(0x7f5234078240 msgr2=0x7f5234104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 shutdown_connections 2024-04-28T12:42:05.039 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:05.042+0000 7f5239a94700 1 -- 172.21.15.171:0/1152514452 wait complete. 2024-04-28T12:42:05.040 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 3 2024-04-28T12:42:05.687 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:05 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:05.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:05 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:05.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:05 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:05.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:05 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/1152514452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-28T12:42:05.688 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:05 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:05.720 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-28T12:42:05.720 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-28T12:42:05.793 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:05 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:05.793 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:05 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:05.793 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:05 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:05.793 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:05 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/1152514452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-28T12:42:05.793 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:05 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:06.130 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mon.c calling monitor election 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mon.a calling monitor election 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: monmap epoch 3 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: last_changed 2024-04-28T12:41:59.303534+0000 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: min_mon_release 19 (squid) 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: election_strategy: 1 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: fsmap 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mgrmap e13: x(active, since 68s) 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: overall HEALTH_OK 2024-04-28T12:42:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:06.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:06.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:06.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:06.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:06.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/1152514452' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-28T12:42:06.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:07.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: mon.b calling monitor election 2024-04-28T12:42:07.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: mon.b calling monitor election 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: mon.a calling monitor election 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: mon.c calling monitor election 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: monmap epoch 3 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: last_changed 2024-04-28T12:41:59.303534+0000 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: min_mon_release 19 (squid) 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: election_strategy: 1 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: fsmap 2024-04-28T12:42:07.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:42:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: mgrmap e13: x(active, since 70s) 2024-04-28T12:42:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: overall HEALTH_OK 2024-04-28T12:42:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:07.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mon.b calling monitor election 2024-04-28T12:42:07.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mon.b calling monitor election 2024-04-28T12:42:07.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mon.a calling monitor election 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mon.c calling monitor election 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: monmap epoch 3 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: last_changed 2024-04-28T12:41:59.303534+0000 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: min_mon_release 19 (squid) 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: election_strategy: 1 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: fsmap 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:42:07.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: mgrmap e13: x(active, since 70s) 2024-04-28T12:42:07.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: overall HEALTH_OK 2024-04-28T12:42:07.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:07.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:07.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:07.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:07.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: mon.b calling monitor election 2024-04-28T12:42:07.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: mon.b calling monitor election 2024-04-28T12:42:07.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: mon.a calling monitor election 2024-04-28T12:42:07.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: mon.c calling monitor election 2024-04-28T12:42:07.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-28T12:42:07.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: monmap epoch 3 2024-04-28T12:42:07.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:42:07.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: last_changed 2024-04-28T12:41:59.303534+0000 2024-04-28T12:42:07.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: created 2024-04-28T12:39:45.237228+0000 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: min_mon_release 19 (squid) 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: election_strategy: 1 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: 0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: 2: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.b 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: fsmap 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: osdmap e4: 0 total, 0 up, 0 in 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: mgrmap e13: x(active, since 70s) 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: overall HEALTH_OK 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:07.304 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:08.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.135+0000 7f187fe00700 1 -- 172.21.15.106:0/28237035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1880102ea0 msgr2=0x7f1880103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:08.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.135+0000 7f187fe00700 1 --2- 172.21.15.106:0/28237035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1880102ea0 0x7f1880103280 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1870009910 tx=0x7f1870009c20 comp rx=0 tx=0).stop 2024-04-28T12:42:08.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.135+0000 7f187fe00700 1 -- 172.21.15.106:0/28237035 shutdown_connections 2024-04-28T12:42:08.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.135+0000 7f187fe00700 1 --2- 172.21.15.106:0/28237035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1880102ea0 0x7f1880103280 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:08.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.135+0000 7f187fe00700 1 -- 172.21.15.106:0/28237035 >> 172.21.15.106:0/28237035 conn(0x7f18800fe770 msgr2=0x7f1880100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:42:08.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.135+0000 7f187fe00700 1 -- 172.21.15.106:0/28237035 shutdown_connections 2024-04-28T12:42:08.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.135+0000 7f187fe00700 1 -- 172.21.15.106:0/28237035 wait complete. 2024-04-28T12:42:08.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.136+0000 7f187fe00700 1 Processor -- start 2024-04-28T12:42:08.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.137+0000 7f187fe00700 1 -- start start 2024-04-28T12:42:08.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.137+0000 7f187fe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1880102ea0 0x7f18801a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:08.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.137+0000 7f187fe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18801a0d30 0x7f188019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:08.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.137+0000 7f187fe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f188019aef0 0x7f188019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:08.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.137+0000 7f187fe00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1880076b00 con 0x7f18801a0d30 2024-04-28T12:42:08.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.137+0000 7f187fe00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1880076980 con 0x7f188019aef0 2024-04-28T12:42:08.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.137+0000 7f187fe00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1880076c80 con 0x7f1880102ea0 2024-04-28T12:42:08.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.137+0000 7f187ea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1880102ea0 0x7f18801a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:08.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.137+0000 7f187e000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18801a0d30 0x7f188019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:08.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187ea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1880102ea0 0x7f18801a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:44888/0 (socket says 172.21.15.106:44888) 2024-04-28T12:42:08.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187ea00700 1 -- 172.21.15.106:0/324228842 learned_addr learned my addr 172.21.15.106:0/324228842 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:42:08.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187f400700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f188019aef0 0x7f188019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:08.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187ea00700 1 -- 172.21.15.106:0/324228842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f188019aef0 msgr2=0x7f188019b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:08.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187ea00700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f188019aef0 0x7f188019b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187ea00700 1 -- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18801a0d30 msgr2=0x7f188019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187ea00700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18801a0d30 0x7f188019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187ea00700 1 -- 172.21.15.106:0/324228842 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18801a76c0 con 0x7f1880102ea0 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187e000700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18801a0d30 0x7f188019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.138+0000 7f187ea00700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1880102ea0 0x7f18801a07f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1870004f80 tx=0x7f1870004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.139+0000 7f1877400700 1 -- 172.21.15.106:0/324228842 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870026070 con 0x7f1880102ea0 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.139+0000 7f1877400700 1 -- 172.21.15.106:0/324228842 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f187000b6f0 con 0x7f1880102ea0 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.139+0000 7f1877400700 1 -- 172.21.15.106:0/324228842 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870009e60 con 0x7f1880102ea0 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.139+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18801a78f0 con 0x7f1880102ea0 2024-04-28T12:42:08.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.139+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18801a7de0 con 0x7f1880102ea0 2024-04-28T12:42:08.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.140+0000 7f1877400700 1 -- 172.21.15.106:0/324228842 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f1870029070 con 0x7f1880102ea0 2024-04-28T12:42:08.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.140+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1880064a80 con 0x7f1880102ea0 2024-04-28T12:42:08.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.141+0000 7f1877400700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1864039ae0 0x7f186403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:08.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.141+0000 7f1877400700 1 -- 172.21.15.106:0/324228842 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f187004d4b0 con 0x7f1880102ea0 2024-04-28T12:42:08.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.141+0000 7f187e000700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1864039ae0 0x7f186403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:08.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.142+0000 7f187e000700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1864039ae0 0x7f186403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f186c009ec0 tx=0x7f186c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.147+0000 7f1877400700 1 -- 172.21.15.106:0/324228842 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1870013070 con 0x7f1880102ea0 2024-04-28T12:42:08.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-28T12:42:08.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: Updating smithi129:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: Updating smithi171:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-28T12:42:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-28T12:42:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: Updating smithi129:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: Updating smithi171:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-28T12:42:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: Updating smithi129:/etc/ceph/ceph.conf 2024-04-28T12:42:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: Updating smithi171:/etc/ceph/ceph.conf 2024-04-28T12:42:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: Updating smithi129:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:42:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: Updating smithi171:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:42:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:42:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:08.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.379+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f18801a80a0 con 0x7f1880102ea0 2024-04-28T12:42:08.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.379+0000 7f1877400700 1 -- 172.21.15.106:0/324228842 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f1870030090 con 0x7f1880102ea0 2024-04-28T12:42:08.376 INFO:teuthology.orchestra.run.smithi106.stdout:# minimal ceph.conf for e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:42:08.376 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2024-04-28T12:42:08.376 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:42:08.376 INFO:teuthology.orchestra.run.smithi106.stdout: mon_host = [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-04-28T12:42:08.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1864039ae0 msgr2=0x7f186403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:08.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1864039ae0 0x7f186403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f186c009ec0 tx=0x7f186c009210 comp rx=0 tx=0).stop 2024-04-28T12:42:08.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1880102ea0 msgr2=0x7f18801a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:08.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1880102ea0 0x7f18801a07f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1870004f80 tx=0x7f1870004fb0 comp rx=0 tx=0).stop 2024-04-28T12:42:08.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 shutdown_connections 2024-04-28T12:42:08.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f188019aef0 0x7f188019b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:08.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1864039ae0 0x7f186403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:08.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1880102ea0 0x7f18801a07f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:08.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 --2- 172.21.15.106:0/324228842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18801a0d30 0x7f188019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:08.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.382+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 >> 172.21.15.106:0/324228842 conn(0x7f18800fe770 msgr2=0x7f1880108a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:42:08.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.383+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 shutdown_connections 2024-04-28T12:42:08.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:08.383+0000 7f187fe00700 1 -- 172.21.15.106:0/324228842 wait complete. 2024-04-28T12:42:09.071 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-28T12:42:09.072 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:42:09.072 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.conf 2024-04-28T12:42:09.154 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:42:09.155 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: Updating smithi106:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:09.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:08 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/324228842' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:09.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: Updating smithi106:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:42:09.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:09.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.249 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.249 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.249 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:42:09.249 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-28T12:42:09.249 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:09.249 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:08 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/324228842' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:09.257 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:42:09.257 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.conf 2024-04-28T12:42:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: Updating smithi106:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-28T12:42:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:09.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:42:09.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-28T12:42:09.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:09.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:08 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/324228842' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:09.288 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:42:09.288 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:42:09.357 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:42:09.358 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-04-28T12:42:09.384 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:42:09.384 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-28T12:42:09.448 INFO:tasks.cephadm:Adding mgr.x on smithi106 2024-04-28T12:42:09.448 INFO:tasks.cephadm:Adding mgr.y on smithi129 2024-04-28T12:42:09.448 INFO:tasks.cephadm:Adding mgr.z on smithi171 2024-04-28T12:42:09.449 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi106=x;smithi129=y;smithi171=z' 2024-04-28T12:42:09.717 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:42:10.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:09 smithi171 ceph-mon[63127]: Reconfiguring mon.a (unknown last config time)... 2024-04-28T12:42:10.171 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:09 smithi171 ceph-mon[63127]: Reconfiguring daemon mon.a on smithi106 2024-04-28T12:42:10.257 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:09 smithi106 ceph-mon[28081]: Reconfiguring mon.a (unknown last config time)... 2024-04-28T12:42:10.258 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:09 smithi106 ceph-mon[28081]: Reconfiguring daemon mon.a on smithi106 2024-04-28T12:42:10.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:09 smithi129 ceph-mon[62418]: Reconfiguring mon.a (unknown last config time)... 2024-04-28T12:42:10.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:09 smithi129 ceph-mon[62418]: Reconfiguring daemon mon.a on smithi106 2024-04-28T12:42:11.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:11 smithi171 ceph-mon[63127]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:11.030 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:11.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:11.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:42:11.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-28T12:42:11.031 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:11.245 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.243+0000 7f2cf47e9700 1 -- 172.21.15.171:0/519140584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec103a50 msgr2=0x7f2cec105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:11.245 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.243+0000 7f2cf47e9700 1 --2- 172.21.15.171:0/519140584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec103a50 0x7f2cec105e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009910 tx=0x7f2ce8009c20 comp rx=0 tx=0).stop 2024-04-28T12:42:11.245 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.243+0000 7f2cf47e9700 1 -- 172.21.15.171:0/519140584 shutdown_connections 2024-04-28T12:42:11.245 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.243+0000 7f2cf47e9700 1 --2- 172.21.15.171:0/519140584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec103a50 0x7f2cec105e40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:11.246 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.243+0000 7f2cf47e9700 1 -- 172.21.15.171:0/519140584 >> 172.21.15.171:0/519140584 conn(0x7f2cec0fd460 msgr2=0x7f2cec0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:42:11.246 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.243+0000 7f2cf47e9700 1 -- 172.21.15.171:0/519140584 shutdown_connections 2024-04-28T12:42:11.246 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.243+0000 7f2cf47e9700 1 -- 172.21.15.171:0/519140584 wait complete. 2024-04-28T12:42:11.246 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf47e9700 1 Processor -- start 2024-04-28T12:42:11.246 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf47e9700 1 -- start start 2024-04-28T12:42:11.247 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf47e9700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cec103a50 0x7f2cec19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:11.247 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf47e9700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec19ae20 0x7f2cec19fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:11.247 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf47e9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cec19b2a0 0x7f2cec1a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:11.247 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf47e9700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec108ea0 con 0x7f2cec19ae20 2024-04-28T12:42:11.247 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf47e9700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec108d20 con 0x7f2cec103a50 2024-04-28T12:42:11.247 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf47e9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec109020 con 0x7f2cec19b2a0 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf2d86700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cec19b2a0 0x7f2cec1a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf2d86700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cec19b2a0 0x7f2cec1a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53564/0 (socket says 172.21.15.171:53564) 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.244+0000 7f2cf2d86700 1 -- 172.21.15.171:0/3707903406 learned_addr learned my addr 172.21.15.171:0/3707903406 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2cf2585700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cec103a50 0x7f2cec19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2cf2d86700 1 -- 172.21.15.171:0/3707903406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cec103a50 msgr2=0x7f2cec19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2cf2d86700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cec103a50 0x7f2cec19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2cf2d86700 1 -- 172.21.15.171:0/3707903406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec19ae20 msgr2=0x7f2cec19fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2cf2d86700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec19ae20 0x7f2cec19fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2cf2d86700 1 -- 172.21.15.171:0/3707903406 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec1a2cf0 con 0x7f2cec19b2a0 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2cf2d86700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cec19b2a0 0x7f2cec1a2420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400ba50 tx=0x7f2ce400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2ce37fe700 1 -- 172.21.15.171:0/3707903406 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4019070 con 0x7f2cec19b2a0 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2ce37fe700 1 -- 172.21.15.171:0/3707903406 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ce4014440 con 0x7f2cec19b2a0 2024-04-28T12:42:11.248 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cec1a2fe0 con 0x7f2cec19b2a0 2024-04-28T12:42:11.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2ce37fe700 1 -- 172.21.15.171:0/3707903406 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4007590 con 0x7f2cec19b2a0 2024-04-28T12:42:11.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.245+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cec1a3520 con 0x7f2cec19b2a0 2024-04-28T12:42:11.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.246+0000 7f2ce37fe700 1 -- 172.21.15.171:0/3707903406 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f2ce401c070 con 0x7f2cec19b2a0 2024-04-28T12:42:11.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.246+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cec064a80 con 0x7f2cec19b2a0 2024-04-28T12:42:11.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.246+0000 7f2ce37fe700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2cd8039ae0 0x7f2cd803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:11.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.246+0000 7f2ce37fe700 1 -- 172.21.15.171:0/3707903406 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2ce404c160 con 0x7f2cec19b2a0 2024-04-28T12:42:11.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.247+0000 7f2cf2585700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2cd8039ae0 0x7f2cd803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:11.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.248+0000 7f2cf2585700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2cd8039ae0 0x7f2cd803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8005190 tx=0x7f2ce801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:11.249 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.251+0000 7f2ce37fe700 1 -- 172.21.15.171:0/3707903406 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2ce40112a0 con 0x7f2cec19b2a0 2024-04-28T12:42:11.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:11 smithi129 ceph-mon[62418]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:11.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:11.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:11.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:42:11.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-28T12:42:11.251 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:10 smithi106 ceph-mon[28081]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:42:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-28T12:42:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:11.453 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.457+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi106=x;smithi129=y;smithi171=z", "target": ["mon-mgr", ""]}) -- 0x7f2cec0638f0 con 0x7f2cd8039ae0 2024-04-28T12:42:11.466 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.470+0000 7f2ce37fe700 1 -- 172.21.15.171:0/3707903406 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2cec0638f0 con 0x7f2cd8039ae0 2024-04-28T12:42:11.467 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mgr update... 2024-04-28T12:42:11.469 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.472+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2cd8039ae0 msgr2=0x7f2cd803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:11.469 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.472+0000 7f2cf47e9700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2cd8039ae0 0x7f2cd803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8005190 tx=0x7f2ce801a040 comp rx=0 tx=0).stop 2024-04-28T12:42:11.469 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.472+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cec19b2a0 msgr2=0x7f2cec1a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:11.469 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.472+0000 7f2cf47e9700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cec19b2a0 0x7f2cec1a2420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400ba50 tx=0x7f2ce400be10 comp rx=0 tx=0).stop 2024-04-28T12:42:11.469 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.473+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 shutdown_connections 2024-04-28T12:42:11.469 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.473+0000 7f2cf47e9700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2cd8039ae0 0x7f2cd803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:11.469 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.473+0000 7f2cf47e9700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cec19b2a0 0x7f2cec1a2420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:11.469 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.473+0000 7f2cf47e9700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cec103a50 0x7f2cec19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:11.470 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.473+0000 7f2cf47e9700 1 --2- 172.21.15.171:0/3707903406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cec19ae20 0x7f2cec19fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:11.470 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.473+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 >> 172.21.15.171:0/3707903406 conn(0x7f2cec0fd460 msgr2=0x7f2cec0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:42:11.470 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.473+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 shutdown_connections 2024-04-28T12:42:11.470 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:42:11.473+0000 7f2cf47e9700 1 -- 172.21.15.171:0/3707903406 wait complete. 2024-04-28T12:42:11.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:12 smithi171 ceph-mon[63127]: Reconfiguring mon.b (monmap changed)... 2024-04-28T12:42:12.000 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:12 smithi171 ceph-mon[63127]: Reconfiguring daemon mon.b on smithi129 2024-04-28T12:42:12.000 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:12.026 DEBUG:teuthology.orchestra.run.smithi129:mgr.y> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.y.service 2024-04-28T12:42:12.029 DEBUG:teuthology.orchestra.run.smithi171:mgr.z> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.z.service 2024-04-28T12:42:12.031 INFO:tasks.cephadm:Deploying OSDs... 2024-04-28T12:42:12.031 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:42:12.032 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-04-28T12:42:12.085 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-28T12:42:12.085 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-04-28T12:42:12.130 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-28T12:42:12.130 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:12.130 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 636 Links: 1 2024-04-28T12:42:12.130 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:12.131 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:12.131 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-28 12:41:11.753763893 +0000 2024-04-28T12:42:12.131 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-28 12:30:57.816969826 +0000 2024-04-28T12:42:12.131 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-28 12:30:57.816969826 +0000 2024-04-28T12:42:12.131 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-28 12:30:57.816969826 +0000 2024-04-28T12:42:12.131 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-28T12:42:12.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:12 smithi106 ceph-mon[28081]: Reconfiguring mon.b (monmap changed)... 2024-04-28T12:42:12.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:12 smithi106 ceph-mon[28081]: Reconfiguring daemon mon.b on smithi129 2024-04-28T12:42:12.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:12.252 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-28T12:42:12.252 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-28T12:42:12.253 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000376913 s, 1.4 MB/s 2024-04-28T12:42:12.256 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-28T12:42:12.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:12 smithi129 ceph-mon[62418]: Reconfiguring mon.b (monmap changed)... 2024-04-28T12:42:12.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:12 smithi129 ceph-mon[62418]: Reconfiguring daemon mon.b on smithi129 2024-04-28T12:42:12.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:12.313 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-04-28T12:42:12.363 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-28T12:42:12.364 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:12.364 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-28T12:42:12.364 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:12.364 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:12.364 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-28 12:41:11.812762497 +0000 2024-04-28T12:42:12.364 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-28 12:30:58.441955313 +0000 2024-04-28T12:42:12.364 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-28 12:30:58.441955313 +0000 2024-04-28T12:42:12.364 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-28 12:30:58.441955313 +0000 2024-04-28T12:42:12.364 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-28T12:42:12.442 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-28T12:42:12.442 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-28T12:42:12.442 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.00034758 s, 1.5 MB/s 2024-04-28T12:42:12.446 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-28T12:42:12.503 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-04-28T12:42:12.549 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-28T12:42:12.549 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:12.550 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 675 Links: 1 2024-04-28T12:42:12.550 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:12.550 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:12.550 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-28 12:41:11.858761408 +0000 2024-04-28T12:42:12.550 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-28 12:30:59.074940615 +0000 2024-04-28T12:42:12.550 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-28 12:30:59.074940615 +0000 2024-04-28T12:42:12.550 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-28 12:30:59.074940615 +0000 2024-04-28T12:42:12.550 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-28T12:42:12.623 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-28T12:42:12.623 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-28T12:42:12.623 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000544239 s, 941 kB/s 2024-04-28T12:42:12.627 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-28T12:42:12.683 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-04-28T12:42:12.733 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-28T12:42:12.733 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:12.733 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-28T12:42:12.733 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:12.733 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:12.733 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-28 12:41:11.906760272 +0000 2024-04-28T12:42:12.733 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-28 12:30:59.700926078 +0000 2024-04-28T12:42:12.733 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-28 12:30:59.700926078 +0000 2024-04-28T12:42:12.733 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-28 12:30:59.700926078 +0000 2024-04-28T12:42:12.733 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-28T12:42:12.813 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-28T12:42:12.813 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-28T12:42:12.813 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000346285 s, 1.5 MB/s 2024-04-28T12:42:12.817 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-28T12:42:12.871 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-04-28T12:42:12.872 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-04-28T12:42:12.889 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-28T12:42:12.890 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-04-28T12:42:12.946 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-28T12:42:12.947 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:12.947 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-28T12:42:12.947 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:12.947 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:12.947 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-28 12:41:58.727662297 +0000 2024-04-28T12:42:12.947 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-28 12:29:55.213857271 +0000 2024-04-28T12:42:12.947 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-28 12:29:55.213857271 +0000 2024-04-28T12:42:12.947 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-28 12:29:55.213857271 +0000 2024-04-28T12:42:12.947 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-28T12:42:13.012 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-28T12:42:13.012 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-28T12:42:13.012 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000223241 s, 2.3 MB/s 2024-04-28T12:42:13.013 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-28T12:42:13.070 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-04-28T12:42:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-28T12:42:13.125 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:13.125 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-28T12:42:13.125 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:13.125 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:13.126 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-28 12:41:58.727662297 +0000 2024-04-28T12:42:13.126 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-28 12:29:55.573849266 +0000 2024-04-28T12:42:13.126 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-28 12:29:55.573849266 +0000 2024-04-28T12:42:13.126 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-28 12:29:55.573849266 +0000 2024-04-28T12:42:13.126 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-28T12:42:13.188 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-28T12:42:13.188 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-28T12:42:13.188 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000226186 s, 2.3 MB/s 2024-04-28T12:42:13.189 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-28T12:42:13.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi106=x;smithi129=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:42:13.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: Saving service mgr spec with placement smithi106=x;smithi129=y;smithi171=z;count:3 2024-04-28T12:42:13.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:13.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:13.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:13.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: Reconfiguring mon.c (monmap changed)... 2024-04-28T12:42:13.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:42:13.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-28T12:42:13.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:13.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:13 smithi171 ceph-mon[63127]: Reconfiguring daemon mon.c on smithi171 2024-04-28T12:42:13.247 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-04-28T12:42:13.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi106=x;smithi129=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:42:13.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: Saving service mgr spec with placement smithi106=x;smithi129=y;smithi171=z;count:3 2024-04-28T12:42:13.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:13.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:13.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:13.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: Reconfiguring mon.c (monmap changed)... 2024-04-28T12:42:13.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:42:13.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-28T12:42:13.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:13.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:13 smithi129 ceph-mon[62418]: Reconfiguring daemon mon.c on smithi171 2024-04-28T12:42:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-28T12:42:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:13.278 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-28T12:42:13.278 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:13.278 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:13.278 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-28 12:41:58.727662297 +0000 2024-04-28T12:42:13.278 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-28 12:29:55.913841705 +0000 2024-04-28T12:42:13.278 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-28 12:29:55.913841705 +0000 2024-04-28T12:42:13.278 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-28 12:29:55.913841705 +0000 2024-04-28T12:42:13.279 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi106=x;smithi129=y;smithi171=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: Saving service mgr spec with placement smithi106=x;smithi129=y;smithi171=z;count:3 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: Reconfiguring mon.c (monmap changed)... 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:13 smithi106 ceph-mon[28081]: Reconfiguring daemon mon.c on smithi171 2024-04-28T12:42:13.340 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-28T12:42:13.340 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-28T12:42:13.340 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000191873 s, 2.7 MB/s 2024-04-28T12:42:13.341 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-28T12:42:13.398 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-04-28T12:42:13.452 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-28T12:42:13.453 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:13.453 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-28T12:42:13.453 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:13.453 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:13.453 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-04-28 12:41:58.728662275 +0000 2024-04-28T12:42:13.453 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-04-28 12:29:56.242834388 +0000 2024-04-28T12:42:13.453 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-04-28 12:29:56.242834388 +0000 2024-04-28T12:42:13.453 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-04-28 12:29:56.242834388 +0000 2024-04-28T12:42:13.453 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-28T12:42:13.515 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-04-28T12:42:13.515 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-04-28T12:42:13.515 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000245695 s, 2.1 MB/s 2024-04-28T12:42:13.516 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-28T12:42:13.572 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-04-28T12:42:13.572 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-04-28T12:42:13.588 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-28T12:42:13.588 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-04-28T12:42:13.644 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-28T12:42:13.644 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:13.644 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-28T12:42:13.644 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:13.644 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:13.644 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-28 12:41:48.981564195 +0000 2024-04-28T12:42:13.644 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-28 12:29:55.210399645 +0000 2024-04-28T12:42:13.644 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-28 12:29:55.210399645 +0000 2024-04-28T12:42:13.644 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-28 12:29:55.210399645 +0000 2024-04-28T12:42:13.645 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-28T12:42:13.706 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-28T12:42:13.706 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-28T12:42:13.706 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000193333 s, 2.6 MB/s 2024-04-28T12:42:13.707 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-28T12:42:13.763 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-04-28T12:42:13.819 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-28T12:42:13.819 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:13.819 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-28T12:42:13.819 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:13.819 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:13.819 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-28 12:41:48.982564175 +0000 2024-04-28T12:42:13.819 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-28 12:29:55.592391972 +0000 2024-04-28T12:42:13.819 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-28 12:29:55.592391972 +0000 2024-04-28T12:42:13.819 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-28 12:29:55.592391972 +0000 2024-04-28T12:42:13.820 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-28T12:42:13.881 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-28T12:42:13.881 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-28T12:42:13.881 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000203168 s, 2.5 MB/s 2024-04-28T12:42:13.883 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-28T12:42:13.938 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-04-28T12:42:13.993 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-28T12:42:13.994 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:13.994 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-28T12:42:13.994 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:13.994 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:13.994 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-28 12:41:48.982564175 +0000 2024-04-28T12:42:13.994 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-28 12:29:55.925385284 +0000 2024-04-28T12:42:13.994 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-28 12:29:55.925385284 +0000 2024-04-28T12:42:13.994 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-28 12:29:55.925385284 +0000 2024-04-28T12:42:13.994 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-28T12:42:14.056 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-28T12:42:14.056 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-28T12:42:14.056 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.0002391 s, 2.1 MB/s 2024-04-28T12:42:14.057 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-28T12:42:14.115 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-04-28T12:42:14.169 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-28T12:42:14.170 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-28T12:42:14.170 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-28T12:42:14.170 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-28T12:42:14.170 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-04-28T12:42:14.170 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-04-28 12:41:48.982564175 +0000 2024-04-28T12:42:14.170 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-04-28 12:29:56.257378615 +0000 2024-04-28T12:42:14.170 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-04-28 12:29:56.257378615 +0000 2024-04-28T12:42:14.170 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-04-28 12:29:56.257378615 +0000 2024-04-28T12:42:14.170 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-28T12:42:14.232 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-04-28T12:42:14.232 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-04-28T12:42:14.232 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000205564 s, 2.5 MB/s 2024-04-28T12:42:14.233 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-28T12:42:14.289 INFO:tasks.cephadm:Deploying osd.0 on smithi106 with /dev/vg_nvme/lv_4... 2024-04-28T12:42:14.289 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-28T12:42:14.716 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:42:14.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:14.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:14.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:14.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:14.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:14.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:14.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-28T12:42:14.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-28T12:42:14.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-28T12:42:14.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:14.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: Deploying daemon mgr.z on smithi171 2024-04-28T12:42:14.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:14 smithi106 ceph-mon[28081]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:14.840 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:14.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:14.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:14.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:14.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:14.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:14.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-28T12:42:14.841 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-28T12:42:14.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-28T12:42:14.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:14.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: Deploying daemon mgr.z on smithi171 2024-04-28T12:42:14.842 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:14 smithi171 ceph-mon[63127]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:15.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: Deploying daemon mgr.z on smithi171 2024-04-28T12:42:15.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:14 smithi129 ceph-mon[62418]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:15.745 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:15 smithi171 systemd[1]: Starting Ceph mgr.z for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:42:16.089 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:15 smithi171 podman[64985]: 2024-04-28 12:42:15.790861799 +0000 UTC m=+0.113128824 container create 8c848c386a46302567c95392d5c01125b8731d8cdbe3c9421743323534fdcdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-28T12:42:16.089 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:15 smithi171 podman[64985]: 2024-04-28 12:42:15.695625551 +0000 UTC m=+0.017892587 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:42:16.089 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 podman[64985]: 2024-04-28 12:42:16.010363912 +0000 UTC m=+0.332630937 container init 8c848c386a46302567c95392d5c01125b8731d8cdbe3c9421743323534fdcdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True) 2024-04-28T12:42:16.089 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 podman[64985]: 2024-04-28 12:42:16.013007584 +0000 UTC m=+0.335274621 container start 8c848c386a46302567c95392d5c01125b8731d8cdbe3c9421743323534fdcdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-28T12:42:16.089 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-mgr[65004]: -- 172.21.15.171:0/3051465223 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a804025a0 con 0x556a803d7000 2024-04-28T12:42:16.089 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 bash[64985]: 8c848c386a46302567c95392d5c01125b8731d8cdbe3c9421743323534fdcdb8 2024-04-28T12:42:16.357 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 systemd[1]: Started Ceph mgr.z for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:42:16.358 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:16.176+0000 7fa664eb3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-28T12:42:16.358 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:16.361+0000 7fa664eb3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:42:16.745 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:16.463+0000 7fa664eb3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-28T12:42:17.173 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:16.770+0000 7fa664eb3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-28T12:42:17.173 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:16.852+0000 7fa664eb3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-28T12:42:17.174 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-mon[63127]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:17.174 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:17.174 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:17.174 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:17.174 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-28T12:42:17.174 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-28T12:42:17.174 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-28T12:42:17.174 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:17.174 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:16 smithi129 ceph-mon[62418]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-28T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-28T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-28T12:42:17.175 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:16 smithi106 ceph-mon[28081]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:17.343 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:17.343 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:17.344 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:17.344 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-28T12:42:17.344 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-28T12:42:17.344 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-28T12:42:17.344 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:17.363 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-28T12:42:17.363 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-28T12:42:17.363 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-28T12:42:17.363 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-28T12:42:17.363 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0339336 s, 309 MB/s 2024-04-28T12:42:17.363 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-28T12:42:18.049 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:17 smithi129 ceph-mon[62418]: Deploying daemon mgr.y on smithi129 2024-04-28T12:42:18.232 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_4 2024-04-28T12:42:18.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:17 smithi171 ceph-mon[63127]: Deploying daemon mgr.y on smithi129 2024-04-28T12:42:18.245 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:18.047+0000 7fa664eb3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-28T12:42:18.245 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:18.214+0000 7fa664eb3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-28T12:42:18.295 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:17 smithi106 ceph-mon[28081]: Deploying daemon mgr.y on smithi129 2024-04-28T12:42:18.430 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 systemd[1]: Starting Ceph mgr.y for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:42:18.652 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:42:18.681 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:18.399+0000 7fa664eb3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-28T12:42:18.681 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:18.684+0000 7fa664eb3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-28T12:42:18.715 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 podman[63781]: 2024-04-28 12:42:18.434346419 +0000 UTC m=+0.121882270 container create 2f3649a5e5d28d47cae563bc662209ac401082082a19c0ff0734f07a448e49a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-28T12:42:18.715 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 podman[63781]: 2024-04-28 12:42:18.338401342 +0000 UTC m=+0.025937211 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:42:18.716 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 podman[63781]: 2024-04-28 12:42:18.636650287 +0000 UTC m=+0.324186139 container init 2f3649a5e5d28d47cae563bc662209ac401082082a19c0ff0734f07a448e49a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-28T12:42:18.716 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 podman[63781]: 2024-04-28 12:42:18.639254282 +0000 UTC m=+0.326790142 container start 2f3649a5e5d28d47cae563bc662209ac401082082a19c0ff0734f07a448e49a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-28T12:42:18.716 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 ceph-mgr[63800]: -- 172.21.15.129:0/3451074350 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f3c8845a0 con 0x561f3c858c00 2024-04-28T12:42:18.716 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 bash[63781]: 2f3649a5e5d28d47cae563bc662209ac401082082a19c0ff0734f07a448e49a6 2024-04-28T12:42:18.974 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 systemd[1]: Started Ceph mgr.y for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:42:18.974 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:18.792+0000 7f1e0dda2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-28T12:42:18.975 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:18 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:18.978+0000 7f1e0dda2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:42:18.993 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:18.776+0000 7fa664eb3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-28T12:42:19.052 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:18 smithi106 ceph-mon[28081]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:19.053 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.053 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.053 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.053 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-mon[63127]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:19.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:18 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.246 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:18.998+0000 7fa664eb3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-28T12:42:19.246 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:19.197+0000 7fa664eb3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:42:19.263 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:19 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:19.083+0000 7f1e0dda2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-28T12:42:19.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:18 smithi129 ceph-mon[62418]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:19.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:18 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:18 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:18 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:18 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:19.763 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:19 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:19.419+0000 7f1e0dda2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-28T12:42:19.764 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:19 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:19.510+0000 7f1e0dda2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-28T12:42:20.244 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:20 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:20.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:20 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:20.245 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:19.840+0000 7fa664eb3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-28T12:42:20.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:20 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:20.581 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:20 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:20.269+0000 7fa664eb3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-28T12:42:20.581 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:20 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:20.381+0000 7fa664eb3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-28T12:42:20.581 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:20 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:20.584+0000 7fa664eb3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-28T12:42:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.729+0000 7f066f000700 1 -- 172.21.15.106:0/3708491307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 msgr2=0x7f06681038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.729+0000 7f066f000700 1 --2- 172.21.15.106:0/3708491307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 0x7f06681038b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0658009920 tx=0x7f0658009c30 comp rx=0 tx=0).stop 2024-04-28T12:42:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.729+0000 7f066f000700 1 -- 172.21.15.106:0/3708491307 shutdown_connections 2024-04-28T12:42:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.729+0000 7f066f000700 1 --2- 172.21.15.106:0/3708491307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0668105030 0x7f066810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.729+0000 7f066f000700 1 --2- 172.21.15.106:0/3708491307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 0x7f06681038b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.729+0000 7f066f000700 1 --2- 172.21.15.106:0/3708491307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668104690 0x7f0668104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.729+0000 7f066f000700 1 -- 172.21.15.106:0/3708491307 >> 172.21.15.106:0/3708491307 conn(0x7f06680fe9f0 msgr2=0x7f0668100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:42:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.729+0000 7f066f000700 1 -- 172.21.15.106:0/3708491307 shutdown_connections 2024-04-28T12:42:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.729+0000 7f066f000700 1 -- 172.21.15.106:0/3708491307 wait complete. 2024-04-28T12:42:20.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.730+0000 7f066f000700 1 Processor -- start 2024-04-28T12:42:20.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.730+0000 7f066f000700 1 -- start start 2024-04-28T12:42:20.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f066f000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 0x7f06680771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:20.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f066f000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0668104690 0x7f0668075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:20.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f066f000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668105030 0x7f0668075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:20.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f066f000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f066810ef70 con 0x7f0668104690 2024-04-28T12:42:20.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f066f000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f066810edf0 con 0x7f0668103490 2024-04-28T12:42:20.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f066f000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f066810f0f0 con 0x7f0668105030 2024-04-28T12:42:20.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f0667e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 0x7f06680771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:20.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f0667e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 0x7f06680771c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:53054/0 (socket says 172.21.15.106:53054) 2024-04-28T12:42:20.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f065f400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0668104690 0x7f0668075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f0667e00700 1 -- 172.21.15.106:0/1800390816 learned_addr learned my addr 172.21.15.106:0/1800390816 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f066d000700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668105030 0x7f0668075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f0667e00700 1 -- 172.21.15.106:0/1800390816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668105030 msgr2=0x7f0668075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f0667e00700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668105030 0x7f0668075d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f0667e00700 1 -- 172.21.15.106:0/1800390816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0668104690 msgr2=0x7f0668075810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f0667e00700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0668104690 0x7f0668075810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.731+0000 7f0667e00700 1 -- 172.21.15.106:0/1800390816 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0668076600 con 0x7f0668103490 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.732+0000 7f0667e00700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 0x7f06680771c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0658009230 tx=0x7f0658005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.732+0000 7f0665600700 1 -- 172.21.15.106:0/1800390816 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658027070 con 0x7f0668103490 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.732+0000 7f0665600700 1 -- 172.21.15.106:0/1800390816 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f065800ba10 con 0x7f0668103490 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.732+0000 7f0665600700 1 -- 172.21.15.106:0/1800390816 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658004930 con 0x7f0668103490 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.732+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06681a37d0 con 0x7f0668103490 2024-04-28T12:42:20.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.732+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681a3bd0 con 0x7f0668103490 2024-04-28T12:42:20.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.733+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066806a110 con 0x7f0668103490 2024-04-28T12:42:20.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.733+0000 7f0665600700 1 -- 172.21.15.106:0/1800390816 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f065802a070 con 0x7f0668103490 2024-04-28T12:42:20.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.734+0000 7f0665600700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f064c039890 0x7f064c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:20.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.734+0000 7f0665600700 1 -- 172.21.15.106:0/1800390816 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f065804d570 con 0x7f0668103490 2024-04-28T12:42:20.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.734+0000 7f065f400700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f064c039890 0x7f064c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:20.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.735+0000 7f065f400700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f064c039890 0x7f064c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0650009750 tx=0x7f0650009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:20.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.739+0000 7f0665600700 1 -- 172.21.15.106:0/1800390816 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0658013030 con 0x7f0668103490 2024-04-28T12:42:20.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:20.982+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0668063860 con 0x7f064c039890 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:20 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:20.785+0000 7f1e0dda2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:20 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:20.953+0000 7f1e0dda2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:21.124+0000 7f1e0dda2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:21.121 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:21.122 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.122 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-28T12:42:21.122 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-28T12:42:21.122 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:21.122 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.122 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:42:21.122 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:42:21.122 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-28T12:42:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:21.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:42:21.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:42:21.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:21.446 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:21.115+0000 7fa664eb3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-28T12:42:21.446 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:21.207+0000 7fa664eb3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-28T12:42:21.446 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:21.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:21.448 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:42:21.448 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:42:21.448 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:21.700 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:21.409+0000 7f1e0dda2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-28T12:42:21.700 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:21.500+0000 7f1e0dda2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-28T12:42:21.700 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:21.704+0000 7f1e0dda2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-28T12:42:21.745 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:21.449+0000 7fa664eb3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-28T12:42:21.745 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:21.580+0000 7fa664eb3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-28T12:42:21.745 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:21.675+0000 7fa664eb3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-28T12:42:22.012 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:21.894+0000 7f1e0dda2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-28T12:42:22.095 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:22 smithi171 ceph-mon[63127]: Reconfiguring mgr.x (unknown last config time)... 2024-04-28T12:42:22.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:22 smithi171 ceph-mon[63127]: Reconfiguring daemon mgr.x on smithi106 2024-04-28T12:42:22.096 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:22 smithi171 ceph-mon[63127]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:42:22.096 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:21.791+0000 7fa664eb3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-28T12:42:22.096 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:22 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:22.099+0000 7fa664eb3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-28T12:42:22.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:22 smithi129 ceph-mon[62418]: Reconfiguring mgr.x (unknown last config time)... 2024-04-28T12:42:22.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:22 smithi129 ceph-mon[62418]: Reconfiguring daemon mgr.x on smithi106 2024-04-28T12:42:22.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:22 smithi129 ceph-mon[62418]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:42:22.300 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:22 smithi106 ceph-mon[28081]: Reconfiguring mgr.x (unknown last config time)... 2024-04-28T12:42:22.300 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:22 smithi106 ceph-mon[28081]: Reconfiguring daemon mgr.x on smithi106 2024-04-28T12:42:22.300 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:22 smithi106 ceph-mon[28081]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:42:22.763 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:22 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:22.489+0000 7f1e0dda2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-28T12:42:22.995 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:22 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:22.698+0000 7fa664eb3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-28T12:42:22.995 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 12:42:22 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z[65000]: 2024-04-28T12:42:22.912+0000 7fa664eb3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-28T12:42:23.203 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:22 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:22.907+0000 7f1e0dda2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-28T12:42:23.203 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:23.003+0000 7f1e0dda2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-28T12:42:23.203 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:23.207+0000 7f1e0dda2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-28T12:42:23.203 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: Standby manager daemon z started 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-28T12:42:23.204 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-mon[62418]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-28T12:42:23.206 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:23.206 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:23.206 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:23.206 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:23.206 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:23.206 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:23.206 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:23.207 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: Standby manager daemon z started 2024-04-28T12:42:23.207 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-28T12:42:23.207 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-28T12:42:23.207 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-28T12:42:23.207 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:23 smithi106 ceph-mon[28081]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-28T12:42:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: Standby manager daemon z started 2024-04-28T12:42:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-28T12:42:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-28T12:42:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-28T12:42:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:23 smithi171 ceph-mon[63127]: from='mgr.? 172.21.15.171:0/3083358513' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-28T12:42:23.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:23.884+0000 7f0665600700 1 -- 172.21.15.106:0/1800390816 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 14) ==== 92603+0+0 (secure 0 0 0) 0x7f065802c440 con 0x7f0668103490 2024-04-28T12:42:24.013 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:23.726+0000 7f1e0dda2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-28T12:42:24.013 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:23.818+0000 7f1e0dda2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-28T12:42:24.396 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:24.059+0000 7f1e0dda2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-28T12:42:24.396 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:24.190+0000 7f1e0dda2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-28T12:42:24.396 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:24.285+0000 7f1e0dda2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-28T12:42:24.396 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:24.399+0000 7f1e0dda2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-28T12:42:25.013 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:24.701+0000 7f1e0dda2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-28T12:42:25.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:24 smithi129 ceph-mon[62418]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-28T12:42:25.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-28T12:42:25.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:24 smithi129 ceph-mon[62418]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:25.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:24 smithi171 ceph-mon[63127]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-28T12:42:25.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-28T12:42:25.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:24 smithi171 ceph-mon[63127]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:25.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:24 smithi106 ceph-mon[28081]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-28T12:42:25.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-28T12:42:25.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:24 smithi106 ceph-mon[28081]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:25.763 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:25.295+0000 7f1e0dda2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-28T12:42:25.763 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y[63796]: 2024-04-28T12:42:25.511+0000 7f1e0dda2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-28T12:42:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:25.915+0000 7f0665600700 1 -- 172.21.15.106:0/1800390816 <== mon.2 v2:172.21.15.129:3300/0 8 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f0658004ad0 con 0x7f0668103490 2024-04-28T12:42:26.230 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-28T12:42:26.230 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: Standby manager daemon y started 2024-04-28T12:42:26.231 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-28T12:42:26.231 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-28T12:42:26.231 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-28T12:42:26.231 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1075211952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0afd8792-8351-40f5-ac93-536a58b5182a"}]: dispatch 2024-04-28T12:42:26.231 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0afd8792-8351-40f5-ac93-536a58b5182a"}]: dispatch 2024-04-28T12:42:26.231 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0afd8792-8351-40f5-ac93-536a58b5182a"}]': finished 2024-04-28T12:42:26.231 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: osdmap e5: 1 total, 0 up, 1 in 2024-04-28T12:42:26.231 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:25 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:26.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-28T12:42:26.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: Standby manager daemon y started 2024-04-28T12:42:26.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-28T12:42:26.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-28T12:42:26.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-28T12:42:26.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1075211952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0afd8792-8351-40f5-ac93-536a58b5182a"}]: dispatch 2024-04-28T12:42:26.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0afd8792-8351-40f5-ac93-536a58b5182a"}]: dispatch 2024-04-28T12:42:26.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0afd8792-8351-40f5-ac93-536a58b5182a"}]': finished 2024-04-28T12:42:26.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: osdmap e5: 1 total, 0 up, 1 in 2024-04-28T12:42:26.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:25 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:26.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-28T12:42:26.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: Standby manager daemon y started 2024-04-28T12:42:26.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-28T12:42:26.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-28T12:42:26.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: from='mgr.? 172.21.15.129:0/344221715' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-28T12:42:26.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1075211952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0afd8792-8351-40f5-ac93-536a58b5182a"}]: dispatch 2024-04-28T12:42:26.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0afd8792-8351-40f5-ac93-536a58b5182a"}]: dispatch 2024-04-28T12:42:26.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0afd8792-8351-40f5-ac93-536a58b5182a"}]': finished 2024-04-28T12:42:26.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: osdmap e5: 1 total, 0 up, 1 in 2024-04-28T12:42:26.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:25 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:27.216 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:26 smithi171 ceph-mon[63127]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-28T12:42:27.216 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:26 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-28T12:42:27.216 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:26 smithi171 ceph-mon[63127]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:27.216 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:26 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3207670872' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:42:27.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:26 smithi129 ceph-mon[62418]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-28T12:42:27.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:26 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-28T12:42:27.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:26 smithi129 ceph-mon[62418]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:27.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:26 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3207670872' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:42:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:26 smithi106 ceph-mon[28081]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-28T12:42:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:26 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-28T12:42:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:26 smithi106 ceph-mon[28081]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:26 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3207670872' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:42:29.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:28 smithi171 ceph-mon[63127]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:29.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:28 smithi129 ceph-mon[62418]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:29.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:28 smithi106 ceph-mon[28081]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:31.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:30 smithi171 ceph-mon[63127]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:31.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:30 smithi129 ceph-mon[62418]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:31.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:30 smithi106 ceph-mon[28081]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:33.152 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:33 smithi106 ceph-mon[28081]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:33.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:33 smithi129 ceph-mon[62418]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:33 smithi171 ceph-mon[63127]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:35.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:35 smithi129 ceph-mon[62418]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:35.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-28T12:42:35.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:35.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:35 smithi106 ceph-mon[28081]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:35.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-28T12:42:35.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:35 smithi171 ceph-mon[63127]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-28T12:42:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:36.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:36 smithi106 ceph-mon[28081]: Deploying daemon osd.0 on smithi106 2024-04-28T12:42:36.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:36 smithi129 ceph-mon[62418]: Deploying daemon osd.0 on smithi106 2024-04-28T12:42:36.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:36 smithi171 ceph-mon[63127]: Deploying daemon osd.0 on smithi106 2024-04-28T12:42:37.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:37 smithi129 ceph-mon[62418]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:37.264 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:37 smithi106 ceph-mon[28081]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:37.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:37 smithi171 ceph-mon[63127]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:39.297 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:39 smithi171 ceph-mon[63127]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:39.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:39 smithi106 ceph-mon[28081]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:39 smithi129 ceph-mon[62418]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:41.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:41 smithi106 ceph-mon[28081]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:41.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:41 smithi171 ceph-mon[63127]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:41.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:41 smithi129 ceph-mon[62418]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:43.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:43 smithi106 ceph-mon[28081]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:43.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:43 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:43.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:43 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:43.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:43 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:43.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:43 smithi171 ceph-mon[63127]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:43.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:43 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:43.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:43 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:43.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:43 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:43.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:43 smithi129 ceph-mon[62418]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:43.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:43 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:43.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:43 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:43.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:43 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:45.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:45 smithi106 ceph-mon[28081]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:45 smithi171 ceph-mon[63127]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:45.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:45 smithi129 ceph-mon[62418]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:47.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:47.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:47.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:47.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:47.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:46 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:47.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:46 smithi106 ceph-mon[28081]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:47.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:46 smithi106 ceph-mon[28081]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-28T12:42:47.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.227+0000 7f0665600700 1 -- 172.21.15.106:0/1800390816 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0668063860 con 0x7f064c039890 2024-04-28T12:42:47.225 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 0 on host 'smithi106' 2024-04-28T12:42:47.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.231+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f064c039890 msgr2=0x7f064c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:47.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.231+0000 7f066f000700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f064c039890 0x7f064c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0650009750 tx=0x7f0650009310 comp rx=0 tx=0).stop 2024-04-28T12:42:47.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.231+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 msgr2=0x7f06680771c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:47.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.231+0000 7f066f000700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 0x7f06680771c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0658009230 tx=0x7f0658005270 comp rx=0 tx=0).stop 2024-04-28T12:42:47.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.232+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 shutdown_connections 2024-04-28T12:42:47.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.232+0000 7f066f000700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f064c039890 0x7f064c03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:47.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.232+0000 7f066f000700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0668105030 0x7f0668075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:47.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.232+0000 7f066f000700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0668103490 0x7f06680771c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:47.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.232+0000 7f066f000700 1 --2- 172.21.15.106:0/1800390816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0668104690 0x7f0668075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:47.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.232+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 >> 172.21.15.106:0/1800390816 conn(0x7f06680fe9f0 msgr2=0x7f0668107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:42:47.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.232+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 shutdown_connections 2024-04-28T12:42:47.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:42:47.232+0000 7f066f000700 1 -- 172.21.15.106:0/1800390816 wait complete. 2024-04-28T12:42:47.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:46 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:47.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:46 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:47.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:46 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:47.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:46 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:47.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:46 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:47.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:46 smithi171 ceph-mon[63127]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:47.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:46 smithi171 ceph-mon[63127]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-28T12:42:47.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:46 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:47.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:46 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:47.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:46 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:47.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:46 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:47.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:46 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:47.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:46 smithi129 ceph-mon[62418]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:47.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:46 smithi129 ceph-mon[62418]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-28T12:42:47.953 DEBUG:teuthology.orchestra.run.smithi106:osd.0> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.0.service 2024-04-28T12:42:47.959 INFO:tasks.cephadm:Deploying osd.1 on smithi129 with /dev/vg_nvme/lv_4... 2024-04-28T12:42:47.959 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-28T12:42:48.192 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.b/config 2024-04-28T12:42:48.201 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:47 smithi106 ceph-mon[28081]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-28T12:42:48.202 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:47 smithi106 ceph-mon[28081]: osdmap e6: 1 total, 0 up, 1 in 2024-04-28T12:42:48.202 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:47 smithi106 ceph-mon[28081]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:42:48.202 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:48.202 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:48.202 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:48.202 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:48.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:47 smithi171 ceph-mon[63127]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-28T12:42:48.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:47 smithi171 ceph-mon[63127]: osdmap e6: 1 total, 0 up, 1 in 2024-04-28T12:42:48.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:47 smithi171 ceph-mon[63127]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:42:48.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:48.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:48.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:48.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:47 smithi129 ceph-mon[62418]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-28T12:42:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:47 smithi129 ceph-mon[62418]: osdmap e6: 1 total, 0 up, 1 in 2024-04-28T12:42:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:47 smithi129 ceph-mon[62418]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:42:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:48.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:42:48.825 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:42:48 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0[42459]: 2024-04-28T12:42:48.714+0000 7f71b9800700 -1 osd.0 0 waiting for initial osdmap 2024-04-28T12:42:48.825 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:42:48 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0[42459]: 2024-04-28T12:42:48.734+0000 7f71af600700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:42:49.233 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:48 smithi129 ceph-mon[62418]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:42:49.233 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:48 smithi129 ceph-mon[62418]: osdmap e7: 1 total, 0 up, 1 in 2024-04-28T12:42:49.233 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:49.234 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:49.234 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:48 smithi129 ceph-mon[62418]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:49.234 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:48 smithi129 ceph-mon[62418]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' 2024-04-28T12:42:49.237 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:48 smithi106 ceph-mon[28081]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:42:49.237 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:48 smithi106 ceph-mon[28081]: osdmap e7: 1 total, 0 up, 1 in 2024-04-28T12:42:49.237 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:49.237 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:49.237 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:48 smithi106 ceph-mon[28081]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:49.237 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:48 smithi106 ceph-mon[28081]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' 2024-04-28T12:42:49.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:48 smithi171 ceph-mon[63127]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:42:49.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:48 smithi171 ceph-mon[63127]: osdmap e7: 1 total, 0 up, 1 in 2024-04-28T12:42:49.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:49.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:49.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:48 smithi171 ceph-mon[63127]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-28T12:42:49.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:48 smithi171 ceph-mon[63127]: from='osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961]' entity='osd.0' 2024-04-28T12:42:49.967 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:49 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:42:49.967 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:49 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:42:49.967 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:49.967 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:49 smithi106 ceph-mon[28081]: osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] boot 2024-04-28T12:42:49.968 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:49 smithi106 ceph-mon[28081]: osdmap e8: 1 total, 1 up, 1 in 2024-04-28T12:42:49.968 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:50.094 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-28T12:42:50.094 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-28T12:42:50.094 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-28T12:42:50.094 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-28T12:42:50.094 INFO:teuthology.orchestra.run.smithi129.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113158 s, 927 MB/s 2024-04-28T12:42:50.094 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-28T12:42:50.138 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:49 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:42:50.138 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:49 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:42:50.138 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:50.138 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:49 smithi129 ceph-mon[62418]: osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] boot 2024-04-28T12:42:50.138 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:49 smithi129 ceph-mon[62418]: osdmap e8: 1 total, 1 up, 1 in 2024-04-28T12:42:50.138 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:50.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:49 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:42:50.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:49 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:42:50.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:50.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:49 smithi171 ceph-mon[63127]: osd.0 [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] boot 2024-04-28T12:42:50.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:49 smithi171 ceph-mon[63127]: osdmap e8: 1 total, 1 up, 1 in 2024-04-28T12:42:50.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-28T12:42:50.710 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_4 2024-04-28T12:42:50.942 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.b/config 2024-04-28T12:42:51.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:50 smithi129 ceph-mon[62418]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:51.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:50 smithi171 ceph-mon[63127]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:51.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:50 smithi106 ceph-mon[28081]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:52.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:51 smithi106 ceph-mon[28081]: osdmap e9: 1 total, 1 up, 1 in 2024-04-28T12:42:52.186 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:51 smithi129 ceph-mon[62418]: osdmap e9: 1 total, 1 up, 1 in 2024-04-28T12:42:52.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:51 smithi171 ceph-mon[63127]: osdmap e9: 1 total, 1 up, 1 in 2024-04-28T12:42:52.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.517+0000 7fb512fa0700 1 -- 172.21.15.129:0/3005911986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c103480 msgr2=0x7fb50c1038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:52.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.517+0000 7fb512fa0700 1 --2- 172.21.15.129:0/3005911986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c103480 0x7fb50c1038a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb5000099c0 tx=0x7fb500009cd0 comp rx=0 tx=0).stop 2024-04-28T12:42:52.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.517+0000 7fb512fa0700 1 -- 172.21.15.129:0/3005911986 shutdown_connections 2024-04-28T12:42:52.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.517+0000 7fb512fa0700 1 --2- 172.21.15.129:0/3005911986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb50c105020 0x7fb50c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:52.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.517+0000 7fb512fa0700 1 --2- 172.21.15.129:0/3005911986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c103480 0x7fb50c1038a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:52.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.517+0000 7fb512fa0700 1 --2- 172.21.15.129:0/3005911986 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c104680 0x7fb50c104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:52.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.517+0000 7fb512fa0700 1 -- 172.21.15.129:0/3005911986 >> 172.21.15.129:0/3005911986 conn(0x7fb50c0fea20 msgr2=0x7fb50c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:42:52.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.517+0000 7fb512fa0700 1 -- 172.21.15.129:0/3005911986 shutdown_connections 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.517+0000 7fb512fa0700 1 -- 172.21.15.129:0/3005911986 wait complete. 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb512fa0700 1 Processor -- start 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb512fa0700 1 -- start start 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb512fa0700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb50c103480 0x7fb50c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb512fa0700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c104680 0x7fb50c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb512fa0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c105020 0x7fb50c1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb512fa0700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10f040 con 0x7fb50c103480 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb512fa0700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10eec0 con 0x7fb50c104680 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb512fa0700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c10f1c0 con 0x7fb50c105020 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb50bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c104680 0x7fb50c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:52.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb50bfff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c104680 0x7fb50c199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36986/0 (socket says 172.21.15.129:36986) 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb50bfff700 1 -- 172.21.15.129:0/2395279620 learned_addr learned my addr 172.21.15.129:0/2395279620 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb50bfff700 1 -- 172.21.15.129:0/2395279620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c105020 msgr2=0x7fb50c1a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.518+0000 7fb51153d700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c105020 0x7fb50c1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb50bfff700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c105020 0x7fb50c1a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb50bfff700 1 -- 172.21.15.129:0/2395279620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb50c103480 msgr2=0x7fb50c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb50bfff700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb50c103480 0x7fb50c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb50bfff700 1 -- 172.21.15.129:0/2395279620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c1a1390 con 0x7fb50c104680 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb51153d700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c105020 0x7fb50c1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb50bfff700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c104680 0x7fb50c199c70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800b530 tx=0x7fb4f800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb509ffb700 1 -- 172.21.15.129:0/2395279620 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8015020 con 0x7fb50c104680 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb509ffb700 1 -- 172.21.15.129:0/2395279620 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f8011ce0 con 0x7fb50c104680 2024-04-28T12:42:52.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c1a1680 con 0x7fb50c104680 2024-04-28T12:42:52.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb509ffb700 1 -- 172.21.15.129:0/2395279620 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8010d50 con 0x7fb50c104680 2024-04-28T12:42:52.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.519+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c1a1c40 con 0x7fb50c104680 2024-04-28T12:42:52.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.520+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50c06a110 con 0x7fb50c104680 2024-04-28T12:42:52.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.521+0000 7fb509ffb700 1 -- 172.21.15.129:0/2395279620 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fb4f8018020 con 0x7fb50c104680 2024-04-28T12:42:52.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.522+0000 7fb509ffb700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb4fc0a3bb0 0x7fb4fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:42:52.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.522+0000 7fb509ffb700 1 -- 172.21.15.129:0/2395279620 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fb4f80ce170 con 0x7fb50c104680 2024-04-28T12:42:52.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.522+0000 7fb510d3c700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb4fc0a3bb0 0x7fb4fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:42:52.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.523+0000 7fb510d3c700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb4fc0a3bb0 0x7fb4fc0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb500003790 tx=0x7fb500009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:42:52.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.527+0000 7fb509ffb700 1 -- 172.21.15.129:0/2395279620 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb4f801f020 con 0x7fb50c104680 2024-04-28T12:42:52.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:42:52.736+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb50c063860 con 0x7fb4fc0a3bb0 2024-04-28T12:42:53.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:53.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:42:53.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:53.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:53.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:42:53.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:42:53.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:53.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:53.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:42:53.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:53.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:53.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:42:53.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:42:53.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:53.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:53.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:42:53.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:53.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:42:53.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:42:53.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:42:53.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:42:53.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:42:54.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:53 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi106 2024-04-28T12:42:54.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:53 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi106 to 17089M 2024-04-28T12:42:54.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:53 smithi129 ceph-mon[62418]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:42:54.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:53 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi106 2024-04-28T12:42:54.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:53 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi106 to 17089M 2024-04-28T12:42:54.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:53 smithi171 ceph-mon[63127]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:42:54.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:53 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi106 2024-04-28T12:42:54.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:53 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi106 to 17089M 2024-04-28T12:42:54.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:53 smithi106 ceph-mon[28081]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:42:55.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:54 smithi171 ceph-mon[63127]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:55.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:54 smithi129 ceph-mon[62418]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:55.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:54 smithi106 ceph-mon[28081]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:56.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:55 smithi129 ceph-mon[62418]: from='client.? 172.21.15.129:0/3292295287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d86a904e-10f6-402f-8107-f2fac6eead2d"}]: dispatch 2024-04-28T12:42:56.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:55 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d86a904e-10f6-402f-8107-f2fac6eead2d"}]: dispatch 2024-04-28T12:42:56.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:55 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d86a904e-10f6-402f-8107-f2fac6eead2d"}]': finished 2024-04-28T12:42:56.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:55 smithi129 ceph-mon[62418]: osdmap e10: 2 total, 1 up, 2 in 2024-04-28T12:42:56.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:55 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:42:56.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:55 smithi171 ceph-mon[63127]: from='client.? 172.21.15.129:0/3292295287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d86a904e-10f6-402f-8107-f2fac6eead2d"}]: dispatch 2024-04-28T12:42:56.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:55 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d86a904e-10f6-402f-8107-f2fac6eead2d"}]: dispatch 2024-04-28T12:42:56.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:55 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d86a904e-10f6-402f-8107-f2fac6eead2d"}]': finished 2024-04-28T12:42:56.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:55 smithi171 ceph-mon[63127]: osdmap e10: 2 total, 1 up, 2 in 2024-04-28T12:42:56.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:55 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:42:56.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:55 smithi106 ceph-mon[28081]: from='client.? 172.21.15.129:0/3292295287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d86a904e-10f6-402f-8107-f2fac6eead2d"}]: dispatch 2024-04-28T12:42:56.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:55 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d86a904e-10f6-402f-8107-f2fac6eead2d"}]: dispatch 2024-04-28T12:42:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:55 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d86a904e-10f6-402f-8107-f2fac6eead2d"}]': finished 2024-04-28T12:42:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:55 smithi106 ceph-mon[28081]: osdmap e10: 2 total, 1 up, 2 in 2024-04-28T12:42:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:42:57.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:56 smithi171 ceph-mon[63127]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:57.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:56 smithi171 ceph-mon[63127]: from='client.? 172.21.15.129:0/3381234901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:42:57.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:56 smithi129 ceph-mon[62418]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:57.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:56 smithi129 ceph-mon[62418]: from='client.? 172.21.15.129:0/3381234901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:42:57.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:56 smithi106 ceph-mon[28081]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:57.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:56 smithi106 ceph-mon[28081]: from='client.? 172.21.15.129:0/3381234901' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:42:59.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:42:58 smithi171 ceph-mon[63127]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:59.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:42:58 smithi129 ceph-mon[62418]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:42:59.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:42:58 smithi106 ceph-mon[28081]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:01.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:01 smithi129 ceph-mon[62418]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:01.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:01 smithi106 ceph-mon[28081]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:01 smithi171 ceph-mon[63127]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:03.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:03 smithi129 ceph-mon[62418]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:03.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:03 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-28T12:43:03.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:03 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:03.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:03 smithi106 ceph-mon[28081]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:03.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:03 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-28T12:43:03.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:03 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:03 smithi171 ceph-mon[63127]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:03 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-28T12:43:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:03 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:04.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:04 smithi129 ceph-mon[62418]: Deploying daemon osd.1 on smithi129 2024-04-28T12:43:04.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:04 smithi106 ceph-mon[28081]: Deploying daemon osd.1 on smithi129 2024-04-28T12:43:04.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:04 smithi171 ceph-mon[63127]: Deploying daemon osd.1 on smithi129 2024-04-28T12:43:05.256 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:05 smithi129 ceph-mon[62418]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:05.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:05 smithi106 ceph-mon[28081]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:05.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:05 smithi171 ceph-mon[63127]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:07.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:07 smithi129 ceph-mon[62418]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:07.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:07 smithi106 ceph-mon[28081]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:07.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:07 smithi171 ceph-mon[63127]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:09.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:09.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:09.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:09.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:08 smithi106 ceph-mon[28081]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:09.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:09.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:09.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:09.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:08 smithi171 ceph-mon[63127]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:08 smithi129 ceph-mon[62418]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:11.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:11 smithi129 ceph-mon[62418]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:11.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:11.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:11.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:11.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:11.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:11 smithi106 ceph-mon[28081]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:11 smithi171 ceph-mon[63127]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:11.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:11.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.671+0000 7fb509ffb700 1 -- 172.21.15.129:0/2395279620 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb50c063860 con 0x7fb4fc0a3bb0 2024-04-28T12:43:11.668 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 1 on host 'smithi129' 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.673+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb4fc0a3bb0 msgr2=0x7fb4fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.673+0000 7fb512fa0700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb4fc0a3bb0 0x7fb4fc0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb500003790 tx=0x7fb500009f10 comp rx=0 tx=0).stop 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.673+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c104680 msgr2=0x7fb50c199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.673+0000 7fb512fa0700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c104680 0x7fb50c199c70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800b530 tx=0x7fb4f800b8f0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.673+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 shutdown_connections 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.673+0000 7fb512fa0700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb4fc0a3bb0 0x7fb4fc0a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.673+0000 7fb512fa0700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c105020 0x7fb50c1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.673+0000 7fb512fa0700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb50c103480 0x7fb50c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.674+0000 7fb512fa0700 1 --2- 172.21.15.129:0/2395279620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb50c104680 0x7fb50c199c70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:11.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.674+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 >> 172.21.15.129:0/2395279620 conn(0x7fb50c0fea20 msgr2=0x7fb50c107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:11.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.674+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 shutdown_connections 2024-04-28T12:43:11.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:43:11.674+0000 7fb512fa0700 1 -- 172.21.15.129:0/2395279620 wait complete. 2024-04-28T12:43:12.341 DEBUG:teuthology.orchestra.run.smithi129:osd.1> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.1.service 2024-04-28T12:43:12.343 INFO:tasks.cephadm:Deploying osd.2 on smithi171 with /dev/vg_nvme/lv_4... 2024-04-28T12:43:12.344 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-28T12:43:12.589 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:43:12.710 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:12.711 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:12.711 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:12.711 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:12 smithi171 ceph-mon[63127]: from='osd.1 [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-28T12:43:12.711 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:12 smithi171 ceph-mon[63127]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-28T12:43:12.711 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:12 smithi171 ceph-mon[63127]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:13.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:13.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:13.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:13.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:12 smithi129 ceph-mon[62418]: from='osd.1 [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-28T12:43:13.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:12 smithi129 ceph-mon[62418]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-28T12:43:13.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:12 smithi129 ceph-mon[62418]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:13.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:13.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:13.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:13.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:12 smithi106 ceph-mon[28081]: from='osd.1 [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-28T12:43:13.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:12 smithi106 ceph-mon[28081]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-28T12:43:13.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:12 smithi106 ceph-mon[28081]: pgmap v68: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:13.848 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:13 smithi129 ceph-mon[62418]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-28T12:43:13.848 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:13 smithi129 ceph-mon[62418]: from='osd.1 [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:43:13.848 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:13 smithi129 ceph-mon[62418]: osdmap e11: 2 total, 1 up, 2 in 2024-04-28T12:43:13.848 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:13.848 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:13 smithi129 ceph-mon[62418]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:43:13.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:13 smithi171 ceph-mon[63127]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-28T12:43:13.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:13 smithi171 ceph-mon[63127]: from='osd.1 [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:43:13.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:13 smithi171 ceph-mon[63127]: osdmap e11: 2 total, 1 up, 2 in 2024-04-28T12:43:13.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:13.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:13 smithi171 ceph-mon[63127]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:43:14.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:13 smithi106 ceph-mon[28081]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-28T12:43:14.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:13 smithi106 ceph-mon[28081]: from='osd.1 [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:43:14.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:13 smithi106 ceph-mon[28081]: osdmap e11: 2 total, 1 up, 2 in 2024-04-28T12:43:14.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:14.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:13 smithi106 ceph-mon[28081]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:43:14.660 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:14 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1[67013]: 2024-04-28T12:43:14.385+0000 7fd9eaba1700 -1 osd.1 0 waiting for initial osdmap 2024-04-28T12:43:14.660 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:14 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1[67013]: 2024-04-28T12:43:14.389+0000 7fd9e298c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:43:14.725 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-28T12:43:14.725 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-28T12:43:14.725 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-28T12:43:14.726 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-28T12:43:14.726 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133374 s, 786 MB/s 2024-04-28T12:43:14.726 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-28T12:43:14.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:14 smithi171 ceph-mon[63127]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:43:14.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:14 smithi171 ceph-mon[63127]: osdmap e12: 2 total, 1 up, 2 in 2024-04-28T12:43:14.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:14.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:14 smithi171 ceph-mon[63127]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:14.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:14 smithi171 ceph-mon[63127]: from='osd.1 ' entity='osd.1' 2024-04-28T12:43:14.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:15.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:14 smithi129 ceph-mon[62418]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:43:15.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:14 smithi129 ceph-mon[62418]: osdmap e12: 2 total, 1 up, 2 in 2024-04-28T12:43:15.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:15.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:14 smithi129 ceph-mon[62418]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:15.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:14 smithi129 ceph-mon[62418]: from='osd.1 ' entity='osd.1' 2024-04-28T12:43:15.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:15.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:14 smithi106 ceph-mon[28081]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:43:15.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:14 smithi106 ceph-mon[28081]: osdmap e12: 2 total, 1 up, 2 in 2024-04-28T12:43:15.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:15.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:14 smithi106 ceph-mon[28081]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-28T12:43:15.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:14 smithi106 ceph-mon[28081]: from='osd.1 ' entity='osd.1' 2024-04-28T12:43:15.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:15.336 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2024-04-28T12:43:15.577 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: osd.1 [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] boot 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: osdmap e13: 2 total, 2 up, 2 in 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:15.988 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:16.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: osd.1 [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] boot 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: osdmap e13: 2 total, 2 up, 2 in 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:16.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: osd.1 [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] boot 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: osdmap e13: 2 total, 2 up, 2 in 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:16.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:16.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:16.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:16.881 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:16 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi129 2024-04-28T12:43:16.881 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:16 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi129 to 17088M 2024-04-28T12:43:16.881 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:16 smithi171 ceph-mon[63127]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:17.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:16 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi129 2024-04-28T12:43:17.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:16 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi129 to 17088M 2024-04-28T12:43:17.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:16 smithi129 ceph-mon[62418]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:17.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:16 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi129 2024-04-28T12:43:17.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:16 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi129 to 17088M 2024-04-28T12:43:17.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:16 smithi106 ceph-mon[28081]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:17.091 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 -- 172.21.15.171:0/3945406867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103f20 msgr2=0x7fda4010a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.091 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 --2- 172.21.15.171:0/3945406867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103f20 0x7fda4010a7d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fda3800b0a0 tx=0x7fda3800b3b0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.091 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 -- 172.21.15.171:0/3945406867 shutdown_connections 2024-04-28T12:43:17.091 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 --2- 172.21.15.171:0/3945406867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103f20 0x7fda4010a7d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.091 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 --2- 172.21.15.171:0/3945406867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda40102380 0x7fda401027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.091 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 --2- 172.21.15.171:0/3945406867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda40103580 0x7fda401039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.091 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 -- 172.21.15.171:0/3945406867 >> 172.21.15.171:0/3945406867 conn(0x7fda400fd920 msgr2=0x7fda400ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:17.091 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 -- 172.21.15.171:0/3945406867 shutdown_connections 2024-04-28T12:43:17.091 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 -- 172.21.15.171:0/3945406867 wait complete. 2024-04-28T12:43:17.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 Processor -- start 2024-04-28T12:43:17.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.095+0000 7fda46afb700 1 -- start start 2024-04-28T12:43:17.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda46afb700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda40102380 0x7fda4006c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda46afb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103580 0x7fda4019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda46afb700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda40103f20 0x7fda401a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda46afb700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda4010df60 con 0x7fda40103f20 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda46afb700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda4010dde0 con 0x7fda40102380 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda46afb700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fda4010e0e0 con 0x7fda40103580 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103580 0x7fda4019e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103580 0x7fda4019e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:38238/0 (socket says 172.21.15.171:38238) 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3ffff700 1 -- 172.21.15.171:0/1376095399 learned_addr learned my addr 172.21.15.171:0/1376095399 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda44897700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda40102380 0x7fda4006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3ffff700 1 -- 172.21.15.171:0/1376095399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda40102380 msgr2=0x7fda4006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3ffff700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda40102380 0x7fda4006c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3ffff700 1 -- 172.21.15.171:0/1376095399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda40103f20 msgr2=0x7fda401a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3ffff700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda40103f20 0x7fda401a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:17.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3ffff700 1 -- 172.21.15.171:0/1376095399 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda401a1400 con 0x7fda40103580 2024-04-28T12:43:17.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda44897700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda40102380 0x7fda4006c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:43:17.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3ffff700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103580 0x7fda4019e4a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b530 tx=0x7fda2c00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:17.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3dffb700 1 -- 172.21.15.171:0/1376095399 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c015020 con 0x7fda40103580 2024-04-28T12:43:17.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3dffb700 1 -- 172.21.15.171:0/1376095399 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2c011ce0 con 0x7fda40103580 2024-04-28T12:43:17.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda3dffb700 1 -- 172.21.15.171:0/1376095399 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c010d20 con 0x7fda40103580 2024-04-28T12:43:17.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.096+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda401a16f0 con 0x7fda40103580 2024-04-28T12:43:17.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.097+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda401a1c60 con 0x7fda40103580 2024-04-28T12:43:17.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.098+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4006a150 con 0x7fda40103580 2024-04-28T12:43:17.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.099+0000 7fda3dffb700 1 -- 172.21.15.171:0/1376095399 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fda2c018020 con 0x7fda40103580 2024-04-28T12:43:17.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.099+0000 7fda3dffb700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fda300a3c30 0x7fda300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:17.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.099+0000 7fda3dffb700 1 -- 172.21.15.171:0/1376095399 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fda2c0ce350 con 0x7fda40103580 2024-04-28T12:43:17.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.100+0000 7fda44897700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fda300a3c30 0x7fda300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:17.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.101+0000 7fda44897700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fda300a3c30 0x7fda300a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fda401033e0 tx=0x7fda34008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:17.100 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.104+0000 7fda3dffb700 1 -- 172.21.15.171:0/1376095399 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fda2c01f020 con 0x7fda40103580 2024-04-28T12:43:17.310 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:17.314+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fda40063860 con 0x7fda300a3c30 2024-04-28T12:43:17.933 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:17 smithi129 ceph-mon[62418]: osdmap e14: 2 total, 2 up, 2 in 2024-04-28T12:43:17.933 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:43:17.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:43:17.934 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:17.947 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:17 smithi171 ceph-mon[63127]: osdmap e14: 2 total, 2 up, 2 in 2024-04-28T12:43:17.948 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:43:17.948 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:43:17.948 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:18.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:17 smithi106 ceph-mon[28081]: osdmap e14: 2 total, 2 up, 2 in 2024-04-28T12:43:18.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:43:18.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:43:18.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:18.877 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:18 smithi171 ceph-mon[63127]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:43:18.878 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:18 smithi171 ceph-mon[63127]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:19.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:18 smithi129 ceph-mon[62418]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:43:19.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:18 smithi129 ceph-mon[62418]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:19.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:18 smithi106 ceph-mon[28081]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:43:19.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:18 smithi106 ceph-mon[28081]: pgmap v75: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:21.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:21 smithi129 ceph-mon[62418]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:21.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:21 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/2270220996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9518ad8-2471-45eb-bf21-ab04fcd7274b"}]: dispatch 2024-04-28T12:43:21.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:21 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9518ad8-2471-45eb-bf21-ab04fcd7274b"}]: dispatch 2024-04-28T12:43:21.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:21 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9518ad8-2471-45eb-bf21-ab04fcd7274b"}]': finished 2024-04-28T12:43:21.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:21 smithi129 ceph-mon[62418]: osdmap e15: 3 total, 2 up, 3 in 2024-04-28T12:43:21.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:21 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:21.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:21 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/2554463073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:43:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:21 smithi106 ceph-mon[28081]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:21 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/2270220996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9518ad8-2471-45eb-bf21-ab04fcd7274b"}]: dispatch 2024-04-28T12:43:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:21 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9518ad8-2471-45eb-bf21-ab04fcd7274b"}]: dispatch 2024-04-28T12:43:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:21 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9518ad8-2471-45eb-bf21-ab04fcd7274b"}]': finished 2024-04-28T12:43:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:21 smithi106 ceph-mon[28081]: osdmap e15: 3 total, 2 up, 3 in 2024-04-28T12:43:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:21 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:21 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/2554463073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:43:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:21 smithi171 ceph-mon[63127]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:21 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/2270220996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9518ad8-2471-45eb-bf21-ab04fcd7274b"}]: dispatch 2024-04-28T12:43:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:21 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9518ad8-2471-45eb-bf21-ab04fcd7274b"}]: dispatch 2024-04-28T12:43:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:21 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9518ad8-2471-45eb-bf21-ab04fcd7274b"}]': finished 2024-04-28T12:43:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:21 smithi171 ceph-mon[63127]: osdmap e15: 3 total, 2 up, 3 in 2024-04-28T12:43:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:21 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:21.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:21 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/2554463073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:43:23.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:23 smithi129 ceph-mon[62418]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:23.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:23 smithi106 ceph-mon[28081]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:23 smithi171 ceph-mon[63127]: pgmap v78: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:25.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:25 smithi129 ceph-mon[62418]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:25.296 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:25 smithi171 ceph-mon[63127]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:25.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:25 smithi106 ceph-mon[28081]: pgmap v79: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:27.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:27 smithi171 ceph-mon[63127]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:27.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:27 smithi106 ceph-mon[28081]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:27 smithi129 ceph-mon[62418]: pgmap v80: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:28.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:28 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-28T12:43:28.315 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:28 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:28 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-28T12:43:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:28 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:28 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-28T12:43:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:28 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:29.218 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:29 smithi171 ceph-mon[63127]: Deploying daemon osd.2 on smithi171 2024-04-28T12:43:29.218 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:29 smithi171 ceph-mon[63127]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:29.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:29 smithi106 ceph-mon[28081]: Deploying daemon osd.2 on smithi171 2024-04-28T12:43:29.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:29 smithi106 ceph-mon[28081]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:29 smithi129 ceph-mon[62418]: Deploying daemon osd.2 on smithi171 2024-04-28T12:43:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:29 smithi129 ceph-mon[62418]: pgmap v81: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:31.195 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:31 smithi171 ceph-mon[63127]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:31.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:31 smithi106 ceph-mon[28081]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:31 smithi129 ceph-mon[62418]: pgmap v82: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:33.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:33 smithi171 ceph-mon[63127]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:33.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:33.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:33.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:33.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:33 smithi106 ceph-mon[28081]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:33.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:33.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:33.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:33 smithi129 ceph-mon[62418]: pgmap v83: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:35.126 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:35 smithi171 ceph-mon[63127]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:35.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:35 smithi106 ceph-mon[28081]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:35 smithi129 ceph-mon[62418]: pgmap v84: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:36.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:36.616 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:36.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:36.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:36.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:36.617 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:36 smithi171 ceph-mon[63127]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:36.696 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.700+0000 7fda3dffb700 1 -- 172.21.15.171:0/1376095399 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda40063860 con 0x7fda300a3c30 2024-04-28T12:43:36.697 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 2 on host 'smithi171' 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fda300a3c30 msgr2=0x7fda300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fda300a3c30 0x7fda300a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fda401033e0 tx=0x7fda34008040 comp rx=0 tx=0).stop 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103580 msgr2=0x7fda4019e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103580 0x7fda4019e4a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b530 tx=0x7fda2c00b8f0 comp rx=0 tx=0).stop 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 shutdown_connections 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda40103f20 0x7fda401a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda40102380 0x7fda4006c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fda300a3c30 0x7fda300a60f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 --2- 172.21.15.171:0/1376095399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda40103580 0x7fda4019e4a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 >> 172.21.15.171:0/1376095399 conn(0x7fda400fd920 msgr2=0x7fda401069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:36.699 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 shutdown_connections 2024-04-28T12:43:36.700 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:43:36.703+0000 7fda46afb700 1 -- 172.21.15.171:0/1376095399 wait complete. 2024-04-28T12:43:36.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:36 smithi106 ceph-mon[28081]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:37.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:37.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:37.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:37.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:37.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:37.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:36 smithi129 ceph-mon[62418]: pgmap v85: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:37.355 DEBUG:teuthology.orchestra.run.smithi171:osd.2> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.2.service 2024-04-28T12:43:37.357 INFO:tasks.cephadm:Deploying osd.3 on smithi106 with /dev/vg_nvme/lv_3... 2024-04-28T12:43:37.357 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-28T12:43:37.727 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:37 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:37.727 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:37 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:37.728 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:37 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:37.728 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:37 smithi106 ceph-mon[28081]: from='osd.2 [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-28T12:43:37.728 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:37 smithi106 ceph-mon[28081]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-28T12:43:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:43:37.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:37 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:37.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:37 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:37.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:37 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:37.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:37 smithi171 ceph-mon[63127]: from='osd.2 [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-28T12:43:37.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:37 smithi171 ceph-mon[63127]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-28T12:43:38.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:37 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:38.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:37 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:38.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:37 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:43:38.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:37 smithi129 ceph-mon[62418]: from='osd.2 [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-28T12:43:38.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:37 smithi129 ceph-mon[62418]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-28T12:43:38.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:38 smithi171 ceph-mon[63127]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-28T12:43:38.809 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:38 smithi171 ceph-mon[63127]: osdmap e16: 3 total, 2 up, 3 in 2024-04-28T12:43:38.810 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:38 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:38.810 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:38 smithi171 ceph-mon[63127]: from='osd.2 [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:43:38.810 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:38 smithi171 ceph-mon[63127]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:43:38.810 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:38 smithi171 ceph-mon[63127]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:38.810 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2[68288]: 2024-04-28T12:43:38.679+0000 7f2c9074e700 -1 osd.2 0 waiting for initial osdmap 2024-04-28T12:43:38.810 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2[68288]: 2024-04-28T12:43:38.684+0000 7f2c8c541700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:43:39.002 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:38 smithi106 ceph-mon[28081]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-28T12:43:39.002 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:38 smithi106 ceph-mon[28081]: osdmap e16: 3 total, 2 up, 3 in 2024-04-28T12:43:39.002 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:38 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:39.002 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:38 smithi106 ceph-mon[28081]: from='osd.2 [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:43:39.002 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:38 smithi106 ceph-mon[28081]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:43:39.003 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:38 smithi106 ceph-mon[28081]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:39.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:38 smithi129 ceph-mon[62418]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-28T12:43:39.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:38 smithi129 ceph-mon[62418]: osdmap e16: 3 total, 2 up, 3 in 2024-04-28T12:43:39.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:38 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:39.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:38 smithi129 ceph-mon[62418]: from='osd.2 [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:43:39.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:38 smithi129 ceph-mon[62418]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:43:39.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:38 smithi129 ceph-mon[62418]: pgmap v87: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:39.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:39 smithi171 ceph-mon[63127]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:43:39.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:39 smithi171 ceph-mon[63127]: osdmap e17: 3 total, 2 up, 3 in 2024-04-28T12:43:39.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:39.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:40.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:39 smithi129 ceph-mon[62418]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:43:40.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:39 smithi129 ceph-mon[62418]: osdmap e17: 3 total, 2 up, 3 in 2024-04-28T12:43:40.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:40.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:40.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:39 smithi106 ceph-mon[28081]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:43:40.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:39 smithi106 ceph-mon[28081]: osdmap e17: 3 total, 2 up, 3 in 2024-04-28T12:43:40.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:40.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-28T12:43:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-28T12:43:40.544 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-28T12:43:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-28T12:43:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0354208 s, 296 MB/s 2024-04-28T12:43:40.544 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 2024-04-28T12:43:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-28T12:43:40.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: osd.2 [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] boot 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: osdmap e18: 3 total, 3 up, 3 in 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: pgmap v90: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:40.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:40.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:40.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:43:40.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:43:40.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:40.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: osd.2 [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] boot 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: osdmap e18: 3 total, 3 up, 3 in 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: pgmap v90: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:41.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:43:41.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: osd.2 [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] boot 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: osdmap e18: 3 total, 3 up, 3 in 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: pgmap v90: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:43:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:41.343 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_3 2024-04-28T12:43:41.765 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:43:41.962 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:41 smithi129 sudo[70701]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-28T12:43:41.962 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:41 smithi129 sudo[70701]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:41.962 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:41 smithi129 sudo[70701]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:41.962 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:41 smithi129 sudo[70701]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:41.962 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:41 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi171 2024-04-28T12:43:41.962 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:41 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-28T12:43:41.963 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:41 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-28T12:43:41.963 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:41 smithi129 ceph-mon[62418]: osdmap e19: 3 total, 3 up, 3 in 2024-04-28T12:43:41.963 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:41 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-28T12:43:41.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:41 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi171 2024-04-28T12:43:41.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:41 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-28T12:43:41.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:41 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-28T12:43:41.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:41 smithi171 ceph-mon[63127]: osdmap e19: 3 total, 3 up, 3 in 2024-04-28T12:43:41.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:41 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-28T12:43:42.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:41 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi171 2024-04-28T12:43:42.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:41 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi171 to 17088M 2024-04-28T12:43:42.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-28T12:43:42.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:41 smithi106 ceph-mon[28081]: osdmap e19: 3 total, 3 up, 3 in 2024-04-28T12:43:42.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-28T12:43:42.075 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:43:41 smithi106 sudo[46925]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-28T12:43:42.075 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:43:41 smithi106 sudo[46925]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.075 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:43:41 smithi106 sudo[46925]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.075 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:43:41 smithi106 sudo[46925]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.075 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:43:41 smithi106 sudo[46929]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-28T12:43:42.075 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:43:41 smithi106 sudo[46929]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.075 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:43:41 smithi106 sudo[46929]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.075 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 12:43:41 smithi106 sudo[46929]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.224 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:41 smithi129 sudo[70704]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-28T12:43:42.224 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:41 smithi129 sudo[70704]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.225 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:41 smithi129 sudo[70704]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.225 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 12:43:41 smithi129 sudo[70704]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.225 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 sudo[70707]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-28T12:43:42.225 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 sudo[70707]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.225 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 sudo[70707]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.225 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 sudo[70707]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.416 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 sudo[46936]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-28T12:43:42.416 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 sudo[46936]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.416 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 sudo[46936]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.416 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 sudo[46936]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.417 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 sudo[46941]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-28T12:43:42.417 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 sudo[46941]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.417 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 sudo[46941]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.417 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 sudo[46941]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72057]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-28T12:43:42.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72057]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72057]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72057]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72060]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-28T12:43:42.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72060]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72060]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72060]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.496 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72051]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-28T12:43:42.496 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72051]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.496 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72051]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.496 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72051]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.496 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72054]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-28T12:43:42.496 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72054]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.496 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72054]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.496 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 12:43:42 smithi171 sudo[72054]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 sudo[70710]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-28T12:43:42.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 sudo[70710]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-28T12:43:42.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 sudo[70710]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-28T12:43:42.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 sudo[70710]: pam_unix(sudo:session): session closed for user root 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: osdmap e20: 3 total, 3 up, 3 in 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:43:42.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:43:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:42 smithi171 ceph-mon[63127]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-28T12:43:43.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-28T12:43:43.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: osdmap e20: 3 total, 3 up, 3 in 2024-04-28T12:43:43.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:43.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-28T12:43:43.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:43:43.015 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:43:43.015 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:43:43.015 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:42 smithi129 ceph-mon[62418]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-28T12:43:43.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-28T12:43:43.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: osdmap e20: 3 total, 3 up, 3 in 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:43:43.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-28T12:43:43.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-28T12:43:43.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-28T12:43:43.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-28T12:43:43.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-28T12:43:43.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:42 smithi106 ceph-mon[28081]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-28T12:43:43.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.950+0000 7fc4c7800700 1 -- 172.21.15.106:0/1360661964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4c0109160 msgr2=0x7fc4c010b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:43.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.950+0000 7fc4c7800700 1 --2- 172.21.15.106:0/1360661964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4c0109160 0x7fc4c010b5d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800b0a0 tx=0x7fc4b800b3b0 comp rx=0 tx=0).stop 2024-04-28T12:43:43.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.950+0000 7fc4c7800700 1 -- 172.21.15.106:0/1360661964 shutdown_connections 2024-04-28T12:43:43.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.950+0000 7fc4c7800700 1 --2- 172.21.15.106:0/1360661964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4c0109160 0x7fc4c010b5d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:43.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.950+0000 7fc4c7800700 1 --2- 172.21.15.106:0/1360661964 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4c0103f00 0x7fc4c01062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:43.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.950+0000 7fc4c7800700 1 --2- 172.21.15.106:0/1360661964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4c0106830 0x7fc4c0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:43.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.950+0000 7fc4c7800700 1 -- 172.21.15.106:0/1360661964 >> 172.21.15.106:0/1360661964 conn(0x7fc4c00fd910 msgr2=0x7fc4c00ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:43:43.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.950+0000 7fc4c7800700 1 -- 172.21.15.106:0/1360661964 shutdown_connections 2024-04-28T12:43:43.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.950+0000 7fc4c7800700 1 -- 172.21.15.106:0/1360661964 wait complete. 2024-04-28T12:43:43.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.951+0000 7fc4c7800700 1 Processor -- start 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c7800700 1 -- start start 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c7800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4c0103f00 0x7fc4c019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c7800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4c0106830 0x7fc4c019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c7800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4c0109160 0x7fc4c01a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c7800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c010eef0 con 0x7fc4c0109160 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c7800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c010ed70 con 0x7fc4c0103f00 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c7800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c010f070 con 0x7fc4c0106830 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c4e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4c0103f00 0x7fc4c019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c4e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4c0103f00 0x7fc4c019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:54400/0 (socket says 172.21.15.106:54400) 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c4e00700 1 -- 172.21.15.106:0/800296009 learned_addr learned my addr 172.21.15.106:0/800296009 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:43:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4c5800700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4c0109160 0x7fc4c01a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.952+0000 7fc4bfe00700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4c0106830 0x7fc4c019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4c4e00700 1 -- 172.21.15.106:0/800296009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4c0106830 msgr2=0x7fc4c019bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4c4e00700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4c0106830 0x7fc4c019bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4c4e00700 1 -- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4c0109160 msgr2=0x7fc4c01a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4c4e00700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4c0109160 0x7fc4c01a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4c4e00700 1 -- 172.21.15.106:0/800296009 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c01a3390 con 0x7fc4c0103f00 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4bfe00700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4c0106830 0x7fc4c019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4c4e00700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4c0103f00 0x7fc4c019b730 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e9c0 tx=0x7fc4b400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4bd600700 1 -- 172.21.15.106:0/800296009 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400cbd0 con 0x7fc4c0103f00 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4bd600700 1 -- 172.21.15.106:0/800296009 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b400cd70 con 0x7fc4c0103f00 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c01a3680 con 0x7fc4c0103f00 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4bd600700 1 -- 172.21.15.106:0/800296009 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4005680 con 0x7fc4c0103f00 2024-04-28T12:43:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.953+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c01a3bc0 con 0x7fc4c0103f00 2024-04-28T12:43:43.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.955+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c006a110 con 0x7fc4c0103f00 2024-04-28T12:43:43.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.956+0000 7fc4bd600700 1 -- 172.21.15.106:0/800296009 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc4b4004750 con 0x7fc4c0103f00 2024-04-28T12:43:43.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.958+0000 7fc4bd600700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc4a80a3cc0 0x7fc4a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:43:43.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.958+0000 7fc4bd600700 1 -- 172.21.15.106:0/800296009 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fc4b40cf2a0 con 0x7fc4c0103f00 2024-04-28T12:43:43.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.958+0000 7fc4bfe00700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc4a80a3cc0 0x7fc4a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:43:43.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.958+0000 7fc4bfe00700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc4a80a3cc0 0x7fc4a80a6180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009cb0 tx=0x7fc4ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:43:43.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:43.962+0000 7fc4bd600700 1 -- 172.21.15.106:0/800296009 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc4b409cfb0 con 0x7fc4c0103f00 2024-04-28T12:43:43.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:43 smithi171 ceph-mon[63127]: osdmap e21: 3 total, 3 up, 3 in 2024-04-28T12:43:43.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:43 smithi171 ceph-mon[63127]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-28T12:43:44.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:43 smithi129 ceph-mon[62418]: osdmap e21: 3 total, 3 up, 3 in 2024-04-28T12:43:44.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:43 smithi129 ceph-mon[62418]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-28T12:43:44.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:43 smithi106 ceph-mon[28081]: osdmap e21: 3 total, 3 up, 3 in 2024-04-28T12:43:44.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:43 smithi106 ceph-mon[28081]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-28T12:43:44.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:43:44.198+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc4c0063860 con 0x7fc4a80a3cc0 2024-04-28T12:43:44.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:44 smithi171 ceph-mon[63127]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:44.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:44 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:43:44.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:44 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:43:44.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:44 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:45.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:44 smithi129 ceph-mon[62418]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:45.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:44 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:43:45.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:44 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:43:45.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:44 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:45.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:44 smithi106 ceph-mon[28081]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:45.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:44 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:43:45.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:44 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:43:45.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:44 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:45.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:45 smithi171 ceph-mon[63127]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:43:46.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:45 smithi129 ceph-mon[62418]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:43:46.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:45 smithi106 ceph-mon[28081]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:43:47.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:46 smithi129 ceph-mon[62418]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:47.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:46 smithi106 ceph-mon[28081]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:47.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:46 smithi171 ceph-mon[63127]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:48 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1626044210' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0"}]: dispatch 2024-04-28T12:43:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:48 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1626044210' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0"}]': finished 2024-04-28T12:43:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:48 smithi129 ceph-mon[62418]: osdmap e22: 4 total, 3 up, 4 in 2024-04-28T12:43:48.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:43:48.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:48 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1626044210' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0"}]: dispatch 2024-04-28T12:43:48.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:48 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1626044210' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0"}]': finished 2024-04-28T12:43:48.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:48 smithi106 ceph-mon[28081]: osdmap e22: 4 total, 3 up, 4 in 2024-04-28T12:43:48.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:43:48.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:48 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1626044210' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0"}]: dispatch 2024-04-28T12:43:48.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:48 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1626044210' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0"}]': finished 2024-04-28T12:43:48.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:48 smithi171 ceph-mon[63127]: osdmap e22: 4 total, 3 up, 4 in 2024-04-28T12:43:48.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:43:49.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:49 smithi129 ceph-mon[62418]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:49.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:49 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3903831305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:43:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:49 smithi106 ceph-mon[28081]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:49.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:49 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3903831305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:43:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:49 smithi171 ceph-mon[63127]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:49.558 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:49 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3903831305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:43:51.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:51 smithi129 ceph-mon[62418]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:51.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:51 smithi106 ceph-mon[28081]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:51.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:51 smithi171 ceph-mon[63127]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:53.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:53 smithi106 ceph-mon[28081]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:53.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:53 smithi171 ceph-mon[63127]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:53.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:53 smithi129 ceph-mon[62418]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:55.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:55 smithi106 ceph-mon[28081]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:55 smithi171 ceph-mon[63127]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:55 smithi129 ceph-mon[62418]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:57.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:57 smithi106 ceph-mon[28081]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:57.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-28T12:43:57.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:57 smithi171 ceph-mon[63127]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-28T12:43:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:57 smithi129 ceph-mon[62418]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-28T12:43:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:43:58.299 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:58 smithi106 ceph-mon[28081]: Deploying daemon osd.3 on smithi106 2024-04-28T12:43:58.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:58 smithi171 ceph-mon[63127]: Deploying daemon osd.3 on smithi106 2024-04-28T12:43:58.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:58 smithi129 ceph-mon[62418]: Deploying daemon osd.3 on smithi106 2024-04-28T12:43:59.151 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:43:59 smithi106 ceph-mon[28081]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:59.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:43:59 smithi171 ceph-mon[63127]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:43:59.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:43:59 smithi129 ceph-mon[62418]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:01.314 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:01 smithi106 ceph-mon[28081]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:01 smithi171 ceph-mon[63127]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:01 smithi129 ceph-mon[62418]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:03.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:03 smithi106 ceph-mon[28081]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:03 smithi171 ceph-mon[63127]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:03 smithi129 ceph-mon[62418]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:05.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:05 smithi106 ceph-mon[28081]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:05.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:05 smithi171 ceph-mon[63127]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:05 smithi129 ceph-mon[62418]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:06.391 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:06.392 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:06.392 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:06.392 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:06 smithi106 ceph-mon[28081]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:06.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:06.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:06.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:06.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:06 smithi171 ceph-mon[63127]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:06.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:06.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:06.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:06.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:06 smithi129 ceph-mon[62418]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:09 smithi129 ceph-mon[62418]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:09 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:09 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:09 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:09 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:09.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:09 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:09.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:09 smithi106 ceph-mon[28081]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:09.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:09 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:09.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:09 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:09.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:09 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:09.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:09 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:09.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:09 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:09.476 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:09 smithi171 ceph-mon[63127]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:09.476 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:09 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:09.476 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:09 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:09.476 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:09 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:09.476 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:09 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:09.476 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:09 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:09.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.837+0000 7fc4bd600700 1 -- 172.21.15.106:0/800296009 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc4c0063860 con 0x7fc4a80a3cc0 2024-04-28T12:44:09.835 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 3 on host 'smithi106' 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc4a80a3cc0 msgr2=0x7fc4a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc4a80a3cc0 0x7fc4a80a6180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009cb0 tx=0x7fc4ac009260 comp rx=0 tx=0).stop 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4c0103f00 msgr2=0x7fc4c019b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4c0103f00 0x7fc4c019b730 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e9c0 tx=0x7fc4b400ecd0 comp rx=0 tx=0).stop 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 shutdown_connections 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4c0109160 0x7fc4c01a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc4c0103f00 0x7fc4c019b730 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc4a80a3cc0 0x7fc4a80a6180 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 --2- 172.21.15.106:0/800296009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4c0106830 0x7fc4c019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 >> 172.21.15.106:0/800296009 conn(0x7fc4c00fd910 msgr2=0x7fc4c0103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.840+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 shutdown_connections 2024-04-28T12:44:09.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:44:09.841+0000 7fc4c7800700 1 -- 172.21.15.106:0/800296009 wait complete. 2024-04-28T12:44:10.277 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:10 smithi106 ceph-mon[28081]: from='osd.3 [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-28T12:44:10.277 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:10 smithi106 ceph-mon[28081]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-28T12:44:10.277 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:10.277 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:10.277 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:10.397 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:10 smithi171 ceph-mon[63127]: from='osd.3 [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-28T12:44:10.397 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:10 smithi171 ceph-mon[63127]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-28T12:44:10.397 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:10.397 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:10.397 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:10 smithi129 ceph-mon[62418]: from='osd.3 [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-28T12:44:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:10 smithi129 ceph-mon[62418]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-28T12:44:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:10.626 DEBUG:teuthology.orchestra.run.smithi106:osd.3> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.3.service 2024-04-28T12:44:10.632 INFO:tasks.cephadm:Deploying osd.4 on smithi129 with /dev/vg_nvme/lv_3... 2024-04-28T12:44:10.632 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-28T12:44:10.879 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.b/config 2024-04-28T12:44:11.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:11 smithi129 ceph-mon[62418]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:11.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:11 smithi129 ceph-mon[62418]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-28T12:44:11.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:11 smithi129 ceph-mon[62418]: from='osd.3 [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:44:11.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:11 smithi129 ceph-mon[62418]: osdmap e23: 4 total, 3 up, 4 in 2024-04-28T12:44:11.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:11.252 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:11 smithi129 ceph-mon[62418]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:44:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:11 smithi106 ceph-mon[28081]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:11 smithi106 ceph-mon[28081]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-28T12:44:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:11 smithi106 ceph-mon[28081]: from='osd.3 [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:44:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:11 smithi106 ceph-mon[28081]: osdmap e23: 4 total, 3 up, 4 in 2024-04-28T12:44:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:11 smithi106 ceph-mon[28081]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:44:11.325 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 12:44:11 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3[49246]: 2024-04-28T12:44:11.054+0000 7f8ed4800700 -1 osd.3 0 waiting for initial osdmap 2024-04-28T12:44:11.325 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 12:44:11 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3[49246]: 2024-04-28T12:44:11.066+0000 7f8ecf600700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:44:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:11 smithi171 ceph-mon[63127]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:11 smithi171 ceph-mon[63127]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-28T12:44:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:11 smithi171 ceph-mon[63127]: from='osd.3 [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:44:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:11 smithi171 ceph-mon[63127]: osdmap e23: 4 total, 3 up, 4 in 2024-04-28T12:44:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:11.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:11 smithi171 ceph-mon[63127]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:44:12.294 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:12 smithi106 ceph-mon[28081]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:44:12.294 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:12 smithi106 ceph-mon[28081]: osdmap e24: 4 total, 3 up, 4 in 2024-04-28T12:44:12.294 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:12.294 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:12.294 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:12 smithi106 ceph-mon[28081]: osd.3 [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] boot 2024-04-28T12:44:12.294 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:12 smithi106 ceph-mon[28081]: osdmap e25: 4 total, 4 up, 4 in 2024-04-28T12:44:12.294 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:12.321 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:12 smithi129 ceph-mon[62418]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:44:12.321 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:12 smithi129 ceph-mon[62418]: osdmap e24: 4 total, 3 up, 4 in 2024-04-28T12:44:12.321 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:12.321 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:12.321 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:12 smithi129 ceph-mon[62418]: osd.3 [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] boot 2024-04-28T12:44:12.322 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:12 smithi129 ceph-mon[62418]: osdmap e25: 4 total, 4 up, 4 in 2024-04-28T12:44:12.322 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:12.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:12 smithi171 ceph-mon[63127]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:44:12.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:12 smithi171 ceph-mon[63127]: osdmap e24: 4 total, 3 up, 4 in 2024-04-28T12:44:12.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:12.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:12.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:12 smithi171 ceph-mon[63127]: osd.3 [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] boot 2024-04-28T12:44:12.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:12 smithi171 ceph-mon[63127]: osdmap e25: 4 total, 4 up, 4 in 2024-04-28T12:44:12.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-28T12:44:12.906 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-28T12:44:12.906 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-28T12:44:12.906 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-28T12:44:12.906 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-28T12:44:12.906 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131464 s, 798 MB/s 2024-04-28T12:44:12.906 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-28T12:44:13.180 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:13 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:44:13.180 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:13 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:44:13.180 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:13 smithi106 ceph-mon[28081]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:13.180 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:13 smithi106 ceph-mon[28081]: osdmap e26: 4 total, 4 up, 4 in 2024-04-28T12:44:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:13 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:44:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:13 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:44:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:13 smithi171 ceph-mon[63127]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:13.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:13 smithi171 ceph-mon[63127]: osdmap e26: 4 total, 4 up, 4 in 2024-04-28T12:44:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:13 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:44:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:13 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:44:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:13 smithi129 ceph-mon[62418]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-28T12:44:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:13 smithi129 ceph-mon[62418]: osdmap e26: 4 total, 4 up, 4 in 2024-04-28T12:44:13.608 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_3 2024-04-28T12:44:13.853 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.b/config 2024-04-28T12:44:15.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:15 smithi129 ceph-mon[62418]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:15.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:15 smithi129 ceph-mon[62418]: osdmap e27: 4 total, 4 up, 4 in 2024-04-28T12:44:15.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:15 smithi106 ceph-mon[28081]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:15.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:15 smithi106 ceph-mon[28081]: osdmap e27: 4 total, 4 up, 4 in 2024-04-28T12:44:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:15 smithi171 ceph-mon[63127]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:15 smithi171 ceph-mon[63127]: osdmap e27: 4 total, 4 up, 4 in 2024-04-28T12:44:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.532+0000 7f9b2df92700 1 -- 172.21.15.129:0/3280377823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b281014f0 msgr2=0x7f9b28101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.532+0000 7f9b2df92700 1 --2- 172.21.15.129:0/3280377823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b281014f0 0x7f9b28101950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009990 tx=0x7f9b1c009ca0 comp rx=0 tx=0).stop 2024-04-28T12:44:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.532+0000 7f9b2df92700 1 -- 172.21.15.129:0/3280377823 shutdown_connections 2024-04-28T12:44:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.532+0000 7f9b2df92700 1 --2- 172.21.15.129:0/3280377823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b28101e90 0x7f9b2810ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.532+0000 7f9b2df92700 1 --2- 172.21.15.129:0/3280377823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b28104a70 0x7f9b28104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.532+0000 7f9b2df92700 1 --2- 172.21.15.129:0/3280377823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b281014f0 0x7f9b28101950 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.532+0000 7f9b2df92700 1 -- 172.21.15.129:0/3280377823 >> 172.21.15.129:0/3280377823 conn(0x7f9b280fd090 msgr2=0x7f9b280ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.532+0000 7f9b2df92700 1 -- 172.21.15.129:0/3280377823 shutdown_connections 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.532+0000 7f9b2df92700 1 -- 172.21.15.129:0/3280377823 wait complete. 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b2df92700 1 Processor -- start 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b2df92700 1 -- start start 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b2df92700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b281014f0 0x7f9b281996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b2df92700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b28101e90 0x7f9b28199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b2df92700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b28104a70 0x7f9b281a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b2df92700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2810f020 con 0x7f9b28101e90 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b2df92700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2810eea0 con 0x7f9b28104a70 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b2df92700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2810f1a0 con 0x7f9b281014f0 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b27fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b28104a70 0x7f9b281a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b27fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b28104a70 0x7f9b281a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41878/0 (socket says 172.21.15.129:41878) 2024-04-28T12:44:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b27fff700 1 -- 172.21.15.129:0/1818779907 learned_addr learned my addr 172.21.15.129:0/1818779907 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b277fe700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b281014f0 0x7f9b281996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b26ffd700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b28101e90 0x7f9b28199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b27fff700 1 -- 172.21.15.129:0/1818779907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b281014f0 msgr2=0x7f9b281996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b27fff700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b281014f0 0x7f9b281996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b27fff700 1 -- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b28101e90 msgr2=0x7f9b28199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b27fff700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b28101e90 0x7f9b28199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.533+0000 7f9b27fff700 1 -- 172.21.15.129:0/1818779907 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b281a12f0 con 0x7f9b28104a70 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.534+0000 7f9b27fff700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b28104a70 0x7f9b281a0c50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000b970 tx=0x7f9b2000bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.534+0000 7f9b24ff9700 1 -- 172.21.15.129:0/1818779907 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20019070 con 0x7f9b28104a70 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.534+0000 7f9b24ff9700 1 -- 172.21.15.129:0/1818779907 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b20014440 con 0x7f9b28104a70 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.534+0000 7f9b24ff9700 1 -- 172.21.15.129:0/1818779907 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000ce60 con 0x7f9b28104a70 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.534+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b281a15e0 con 0x7f9b28104a70 2024-04-28T12:44:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.534+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b281a1b50 con 0x7f9b28104a70 2024-04-28T12:44:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.535+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b2806a110 con 0x7f9b28104a70 2024-04-28T12:44:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.536+0000 7f9b24ff9700 1 -- 172.21.15.129:0/1818779907 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9b2001c070 con 0x7f9b28104a70 2024-04-28T12:44:15.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.536+0000 7f9b24ff9700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9b100a3c40 0x7f9b100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:44:15.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.537+0000 7f9b24ff9700 1 -- 172.21.15.129:0/1818779907 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f9b200ce5e0 con 0x7f9b28104a70 2024-04-28T12:44:15.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.537+0000 7f9b277fe700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9b100a3c40 0x7f9b100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:44:15.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.537+0000 7f9b277fe700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9b100a3c40 0x7f9b100a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b18006fd0 tx=0x7f9b18008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:44:15.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.541+0000 7f9b24ff9700 1 -- 172.21.15.129:0/1818779907 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9b2009c1e0 con 0x7f9b28104a70 2024-04-28T12:44:15.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:15.749+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9b28063860 con 0x7f9b100a3c40 2024-04-28T12:44:16.276 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:16 smithi106 ceph-mon[28081]: osdmap e28: 4 total, 4 up, 4 in 2024-04-28T12:44:16.276 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:44:16.276 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:44:16.276 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:16.390 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:16 smithi129 ceph-mon[62418]: osdmap e28: 4 total, 4 up, 4 in 2024-04-28T12:44:16.391 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:44:16.391 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:44:16.391 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:16.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:16 smithi171 ceph-mon[63127]: osdmap e28: 4 total, 4 up, 4 in 2024-04-28T12:44:16.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:44:16.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:44:16.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:44:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-28T12:44:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: osdmap e29: 4 total, 4 up, 4 in 2024-04-28T12:44:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:17.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:17.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.433 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:44:17.433 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-28T12:44:17.433 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: osdmap e29: 4 total, 4 up, 4 in 2024-04-28T12:44:17.433 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.433 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.433 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:17.433 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:17.433 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.433 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:17.434 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:17.434 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:44:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-28T12:44:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: osdmap e29: 4 total, 4 up, 4 in 2024-04-28T12:44:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:17.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:17.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:17.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:17.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:17.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:18.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:18 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi106 2024-04-28T12:44:18.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:18 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-28T12:44:18.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:18 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi106 2024-04-28T12:44:18.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:18 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-28T12:44:18.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:18 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi106 2024-04-28T12:44:18.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:18 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-28T12:44:19.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:19 smithi129 ceph-mon[62418]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-28T12:44:19.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:19 smithi129 ceph-mon[62418]: from='client.? 172.21.15.129:0/309832421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "505898bb-0925-4988-a527-d39888287e0e"}]: dispatch 2024-04-28T12:44:19.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:19 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "505898bb-0925-4988-a527-d39888287e0e"}]: dispatch 2024-04-28T12:44:19.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:19 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "505898bb-0925-4988-a527-d39888287e0e"}]': finished 2024-04-28T12:44:19.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:19 smithi129 ceph-mon[62418]: osdmap e30: 5 total, 4 up, 5 in 2024-04-28T12:44:19.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:19 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:19 smithi171 ceph-mon[63127]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-28T12:44:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:19 smithi171 ceph-mon[63127]: from='client.? 172.21.15.129:0/309832421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "505898bb-0925-4988-a527-d39888287e0e"}]: dispatch 2024-04-28T12:44:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:19 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "505898bb-0925-4988-a527-d39888287e0e"}]: dispatch 2024-04-28T12:44:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:19 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "505898bb-0925-4988-a527-d39888287e0e"}]': finished 2024-04-28T12:44:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:19 smithi171 ceph-mon[63127]: osdmap e30: 5 total, 4 up, 5 in 2024-04-28T12:44:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:19 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:19.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:19 smithi106 ceph-mon[28081]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-28T12:44:19.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:19 smithi106 ceph-mon[28081]: from='client.? 172.21.15.129:0/309832421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "505898bb-0925-4988-a527-d39888287e0e"}]: dispatch 2024-04-28T12:44:19.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:19 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "505898bb-0925-4988-a527-d39888287e0e"}]: dispatch 2024-04-28T12:44:19.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:19 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "505898bb-0925-4988-a527-d39888287e0e"}]': finished 2024-04-28T12:44:19.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:19 smithi106 ceph-mon[28081]: osdmap e30: 5 total, 4 up, 5 in 2024-04-28T12:44:19.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:19 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:20.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:20 smithi171 ceph-mon[63127]: from='client.? 172.21.15.129:0/2369972513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:44:20.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:20 smithi129 ceph-mon[62418]: from='client.? 172.21.15.129:0/2369972513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:44:20.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:20 smithi106 ceph-mon[28081]: from='client.? 172.21.15.129:0/2369972513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:44:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:21 smithi171 ceph-mon[63127]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-28T12:44:21.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:21 smithi129 ceph-mon[62418]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-28T12:44:21.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:21 smithi106 ceph-mon[28081]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-28T12:44:23.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:23 smithi129 ceph-mon[62418]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-28T12:44:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:23 smithi171 ceph-mon[63127]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-28T12:44:23.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:23 smithi106 ceph-mon[28081]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-28T12:44:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:25 smithi171 ceph-mon[63127]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:25.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:25 smithi129 ceph-mon[62418]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:25.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:25 smithi106 ceph-mon[28081]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:26 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-28T12:44:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:26 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:26 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-28T12:44:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:26 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:26.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:26 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-28T12:44:26.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:26 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:27.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:27 smithi171 ceph-mon[63127]: Deploying daemon osd.4 on smithi129 2024-04-28T12:44:27.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:27 smithi171 ceph-mon[63127]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:27 smithi129 ceph-mon[62418]: Deploying daemon osd.4 on smithi129 2024-04-28T12:44:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:27 smithi129 ceph-mon[62418]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:27.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:27 smithi106 ceph-mon[28081]: Deploying daemon osd.4 on smithi129 2024-04-28T12:44:27.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:27 smithi106 ceph-mon[28081]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:29.470 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:29 smithi129 ceph-mon[62418]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:29 smithi171 ceph-mon[63127]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:29.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:29 smithi106 ceph-mon[28081]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:31.445 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:31 smithi129 ceph-mon[62418]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:31.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:31 smithi171 ceph-mon[63127]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:31.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:31 smithi106 ceph-mon[28081]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:32 smithi171 ceph-mon[63127]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:32 smithi129 ceph-mon[62418]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:33.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:33.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:33.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:33.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:32 smithi106 ceph-mon[28081]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:35.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:35 smithi106 ceph-mon[28081]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:35.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:35.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:35.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:35.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:35.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:35.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:35 smithi129 ceph-mon[62418]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:35.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:35.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:35.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:35.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:35.428 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:35 smithi171 ceph-mon[63127]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:35.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.939+0000 7f9b24ff9700 1 -- 172.21.15.129:0/1818779907 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b28063860 con 0x7f9b100a3c40 2024-04-28T12:44:35.935 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 4 on host 'smithi129' 2024-04-28T12:44:35.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9b100a3c40 msgr2=0x7f9b100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:35.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9b100a3c40 0x7f9b100a6100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9b18006fd0 tx=0x7f9b18008040 comp rx=0 tx=0).stop 2024-04-28T12:44:35.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b28104a70 msgr2=0x7f9b281a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:35.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b28104a70 0x7f9b281a0c50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000b970 tx=0x7f9b2000bd30 comp rx=0 tx=0).stop 2024-04-28T12:44:35.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 shutdown_connections 2024-04-28T12:44:35.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b28104a70 0x7f9b281a0c50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:35.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9b100a3c40 0x7f9b100a6100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:35.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b281014f0 0x7f9b281996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:35.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 --2- 172.21.15.129:0/1818779907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b28101e90 0x7f9b28199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:35.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 >> 172.21.15.129:0/1818779907 conn(0x7f9b280fd090 msgr2=0x7f9b28107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:44:35.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 shutdown_connections 2024-04-28T12:44:35.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:44:35.941+0000 7f9b2df92700 1 -- 172.21.15.129:0/1818779907 wait complete. 2024-04-28T12:44:36.248 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:36 smithi129 ceph-mon[62418]: from='osd.4 [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-28T12:44:36.248 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:36 smithi129 ceph-mon[62418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-28T12:44:36.248 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:36.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:36.249 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:36.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:36 smithi106 ceph-mon[28081]: from='osd.4 [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-28T12:44:36.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:36 smithi106 ceph-mon[28081]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-28T12:44:36.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:36.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:36.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:36.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:36 smithi171 ceph-mon[63127]: from='osd.4 [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-28T12:44:36.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:36 smithi171 ceph-mon[63127]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-28T12:44:36.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:36.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:36.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:36.575 DEBUG:teuthology.orchestra.run.smithi129:osd.4> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.4.service 2024-04-28T12:44:36.577 INFO:tasks.cephadm:Deploying osd.5 on smithi171 with /dev/vg_nvme/lv_3... 2024-04-28T12:44:36.577 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-28T12:44:36.824 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:44:37.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:37 smithi171 ceph-mon[63127]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:37.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:37 smithi171 ceph-mon[63127]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-28T12:44:37.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:37 smithi171 ceph-mon[63127]: from='osd.4 [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:44:37.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:37 smithi171 ceph-mon[63127]: osdmap e31: 5 total, 4 up, 5 in 2024-04-28T12:44:37.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:37 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:37.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:37 smithi171 ceph-mon[63127]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:44:37.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:37 smithi129 ceph-mon[62418]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:37.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:37 smithi129 ceph-mon[62418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-28T12:44:37.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:37 smithi129 ceph-mon[62418]: from='osd.4 [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:44:37.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:37 smithi129 ceph-mon[62418]: osdmap e31: 5 total, 4 up, 5 in 2024-04-28T12:44:37.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:37 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:37.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:37 smithi129 ceph-mon[62418]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:44:37.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:37 smithi106 ceph-mon[28081]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:37.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:37 smithi106 ceph-mon[28081]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-28T12:44:37.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:37 smithi106 ceph-mon[28081]: from='osd.4 [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:44:37.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:37 smithi106 ceph-mon[28081]: osdmap e31: 5 total, 4 up, 5 in 2024-04-28T12:44:37.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:37 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:37.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:37 smithi106 ceph-mon[28081]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:44:37.807 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 12:44:37 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4[73514]: 2024-04-28T12:44:37.708+0000 7f0ae0110700 -1 osd.4 0 waiting for initial osdmap 2024-04-28T12:44:37.807 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 12:44:37 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4[73514]: 2024-04-28T12:44:37.714+0000 7f0ad7efb700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:44:38.104 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:38 smithi129 ceph-mon[62418]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:44:38.104 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:38 smithi129 ceph-mon[62418]: osdmap e32: 5 total, 4 up, 5 in 2024-04-28T12:44:38.104 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:38 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:38.104 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:38 smithi129 ceph-mon[62418]: from='osd.4 ' entity='osd.4' 2024-04-28T12:44:38.104 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:38 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:38.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:38 smithi106 ceph-mon[28081]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:44:38.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:38 smithi106 ceph-mon[28081]: osdmap e32: 5 total, 4 up, 5 in 2024-04-28T12:44:38.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:38 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:38.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:38 smithi106 ceph-mon[28081]: from='osd.4 ' entity='osd.4' 2024-04-28T12:44:38.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:38 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:38 smithi171 ceph-mon[63127]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:44:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:38 smithi171 ceph-mon[63127]: osdmap e32: 5 total, 4 up, 5 in 2024-04-28T12:44:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:38 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:38 smithi171 ceph-mon[63127]: from='osd.4 ' entity='osd.4' 2024-04-28T12:44:38.363 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:38 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:38.950 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-28T12:44:38.951 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-28T12:44:38.951 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-28T12:44:38.951 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-28T12:44:38.951 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131326 s, 798 MB/s 2024-04-28T12:44:38.951 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-28T12:44:39.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:39 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:44:39.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:39 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:44:39.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:39 smithi129 ceph-mon[62418]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:39.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:39 smithi129 ceph-mon[62418]: osd.4 [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] boot 2024-04-28T12:44:39.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:39 smithi129 ceph-mon[62418]: osdmap e33: 5 total, 5 up, 5 in 2024-04-28T12:44:39.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:39.316 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:39 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:44:39.317 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:39 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:44:39.317 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:39 smithi171 ceph-mon[63127]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:39.317 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:39 smithi171 ceph-mon[63127]: osd.4 [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] boot 2024-04-28T12:44:39.317 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:39 smithi171 ceph-mon[63127]: osdmap e33: 5 total, 5 up, 5 in 2024-04-28T12:44:39.317 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:39.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:39 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:44:39.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:39 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:44:39.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:39 smithi106 ceph-mon[28081]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-28T12:44:39.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:39 smithi106 ceph-mon[28081]: osd.4 [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] boot 2024-04-28T12:44:39.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:39 smithi106 ceph-mon[28081]: osdmap e33: 5 total, 5 up, 5 in 2024-04-28T12:44:39.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-28T12:44:39.620 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2024-04-28T12:44:39.867 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:44:40.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi129 2024-04-28T12:44:40.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:40.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi129 to 8544M 2024-04-28T12:44:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:40.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:40 smithi171 ceph-mon[63127]: osdmap e34: 5 total, 5 up, 5 in 2024-04-28T12:44:41.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi129 2024-04-28T12:44:41.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:41.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:41.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:41.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi129 to 8544M 2024-04-28T12:44:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:41.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:40 smithi129 ceph-mon[62418]: osdmap e34: 5 total, 5 up, 5 in 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi129 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi129 to 8544M 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:41.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:40 smithi106 ceph-mon[28081]: osdmap e34: 5 total, 5 up, 5 in 2024-04-28T12:44:41.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.534+0000 7f03387a1700 1 -- 172.21.15.171:0/2334861657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330103480 msgr2=0x7f03301038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:41.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.534+0000 7f03387a1700 1 --2- 172.21.15.171:0/2334861657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330103480 0x7f03301038a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f032c009990 tx=0x7f032c009ca0 comp rx=0 tx=0).stop 2024-04-28T12:44:41.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 -- 172.21.15.171:0/2334861657 shutdown_connections 2024-04-28T12:44:41.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 --2- 172.21.15.171:0/2334861657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0330105020 0x7f033010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 --2- 172.21.15.171:0/2334861657 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330103480 0x7f03301038a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 --2- 172.21.15.171:0/2334861657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0330104680 0x7f0330104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 -- 172.21.15.171:0/2334861657 >> 172.21.15.171:0/2334861657 conn(0x7f03300fea20 msgr2=0x7f0330100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 -- 172.21.15.171:0/2334861657 shutdown_connections 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 -- 172.21.15.171:0/2334861657 wait complete. 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 Processor -- start 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 -- start start 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0330103480 0x7f0330077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0330104680 0x7f0330075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330105020 0x7f0330075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f033010efa0 con 0x7f0330103480 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f033010ee20 con 0x7f0330104680 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.535+0000 7f03387a1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f033010f120 con 0x7f0330105020 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0336d3e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330105020 0x7f0330075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:44:41.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0336d3e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330105020 0x7f0330075d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:53598/0 (socket says 172.21.15.171:53598) 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0335d3c700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0330104680 0x7f0330075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0336d3e700 1 -- 172.21.15.171:0/2281187995 learned_addr learned my addr 172.21.15.171:0/2281187995 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0336d3e700 1 -- 172.21.15.171:0/2281187995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0330104680 msgr2=0x7f0330075810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0336d3e700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0330104680 0x7f0330075810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0336d3e700 1 -- 172.21.15.171:0/2281187995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0330103480 msgr2=0x7f0330077170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0336d3e700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0330103480 0x7f0330077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0336d3e700 1 -- 172.21.15.171:0/2281187995 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0330076620 con 0x7f0330105020 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f0336d3e700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330105020 0x7f0330075d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f032800b970 tx=0x7f032800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f03277fe700 1 -- 172.21.15.171:0/2281187995 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328019070 con 0x7f0330105020 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f03277fe700 1 -- 172.21.15.171:0/2281187995 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f032800cde0 con 0x7f0330105020 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f03277fe700 1 -- 172.21.15.171:0/2281187995 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328007370 con 0x7f0330105020 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03301a37f0 con 0x7f0330105020 2024-04-28T12:44:41.534 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.536+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301a3d00 con 0x7f0330105020 2024-04-28T12:44:41.535 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.537+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033006a110 con 0x7f0330105020 2024-04-28T12:44:41.535 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.538+0000 7f03277fe700 1 -- 172.21.15.171:0/2281187995 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f032801c070 con 0x7f0330105020 2024-04-28T12:44:41.535 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.539+0000 7f03277fe700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f031c0a3cc0 0x7f031c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:44:41.535 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.539+0000 7f03277fe700 1 -- 172.21.15.171:0/2281187995 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f03280ce7f0 con 0x7f0330105020 2024-04-28T12:44:41.535 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.539+0000 7f033653d700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f031c0a3cc0 0x7f031c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:44:41.536 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.540+0000 7f033653d700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f031c0a3cc0 0x7f031c0a6180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b5c0 tx=0x7f032c003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:44:41.539 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.544+0000 7f03277fe700 1 -- 172.21.15.171:0/2281187995 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f032809c240 con 0x7f0330105020 2024-04-28T12:44:41.745 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:44:41.749+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0330063860 con 0x7f031c0a3cc0 2024-04-28T12:44:42.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:44:42.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:44:42.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:41 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:42.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:41 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:44:42.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:41 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:44:42.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:41 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:42.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:41 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:44:42.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:41 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:44:42.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:41 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:42.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:42 smithi171 ceph-mon[63127]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:44:42.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:42 smithi171 ceph-mon[63127]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:43.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:42 smithi106 ceph-mon[28081]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:44:43.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:42 smithi106 ceph-mon[28081]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:43.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:42 smithi129 ceph-mon[62418]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:44:43.266 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:42 smithi129 ceph-mon[62418]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:45.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:45 smithi171 ceph-mon[63127]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:45.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:45 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/617456911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4836336-5952-4041-abaf-800c8058d713"}]: dispatch 2024-04-28T12:44:45.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:45 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4836336-5952-4041-abaf-800c8058d713"}]: dispatch 2024-04-28T12:44:45.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:45 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4836336-5952-4041-abaf-800c8058d713"}]': finished 2024-04-28T12:44:45.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:45 smithi171 ceph-mon[63127]: osdmap e35: 6 total, 5 up, 6 in 2024-04-28T12:44:45.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:45 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:44:45.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:45 smithi106 ceph-mon[28081]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:45 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/617456911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4836336-5952-4041-abaf-800c8058d713"}]: dispatch 2024-04-28T12:44:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:45 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4836336-5952-4041-abaf-800c8058d713"}]: dispatch 2024-04-28T12:44:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:45 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4836336-5952-4041-abaf-800c8058d713"}]': finished 2024-04-28T12:44:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:45 smithi106 ceph-mon[28081]: osdmap e35: 6 total, 5 up, 6 in 2024-04-28T12:44:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:45 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:44:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:45 smithi129 ceph-mon[62418]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:45 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/617456911' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4836336-5952-4041-abaf-800c8058d713"}]: dispatch 2024-04-28T12:44:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:45 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4836336-5952-4041-abaf-800c8058d713"}]: dispatch 2024-04-28T12:44:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:45 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4836336-5952-4041-abaf-800c8058d713"}]': finished 2024-04-28T12:44:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:45 smithi129 ceph-mon[62418]: osdmap e35: 6 total, 5 up, 6 in 2024-04-28T12:44:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:45 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:44:46.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:46 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/1354995336' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:44:46.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:46 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/1354995336' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:44:46.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:46 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/1354995336' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:44:47.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:47 smithi106 ceph-mon[28081]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:47.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:47 smithi171 ceph-mon[63127]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:47 smithi129 ceph-mon[62418]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:49.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:49 smithi171 ceph-mon[63127]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:49.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:49 smithi129 ceph-mon[62418]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:49 smithi106 ceph-mon[28081]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:51.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:51 smithi171 ceph-mon[63127]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:51.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:51 smithi106 ceph-mon[28081]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:51 smithi129 ceph-mon[62418]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:52.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-28T12:44:52.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-28T12:44:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-28T12:44:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:44:53.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:53 smithi171 ceph-mon[63127]: Deploying daemon osd.5 on smithi171 2024-04-28T12:44:53.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:53 smithi171 ceph-mon[63127]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:53.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:53 smithi106 ceph-mon[28081]: Deploying daemon osd.5 on smithi171 2024-04-28T12:44:53.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:53 smithi106 ceph-mon[28081]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:53.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:53 smithi129 ceph-mon[62418]: Deploying daemon osd.5 on smithi171 2024-04-28T12:44:53.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:53 smithi129 ceph-mon[62418]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:55.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:55 smithi171 ceph-mon[63127]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:55.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:55 smithi106 ceph-mon[28081]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:55 smithi129 ceph-mon[62418]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:57.109 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:57 smithi171 ceph-mon[63127]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:57.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:57 smithi106 ceph-mon[28081]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:57 smithi129 ceph-mon[62418]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:58.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:58.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:58.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:58.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:44:58 smithi171 ceph-mon[63127]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:59.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:59.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:59.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:59.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:44:58 smithi129 ceph-mon[62418]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:44:59.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:59.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:44:59.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:44:59.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:44:58 smithi106 ceph-mon[28081]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:01 smithi106 ceph-mon[28081]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:01.345 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:01 smithi171 ceph-mon[63127]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:01.345 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:01.345 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:01.345 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:01.345 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:01.346 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:01 smithi129 ceph-mon[62418]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:01.730 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.733+0000 7f03277fe700 1 -- 172.21.15.171:0/2281187995 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0330063860 con 0x7f031c0a3cc0 2024-04-28T12:45:01.730 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 5 on host 'smithi171' 2024-04-28T12:45:01.731 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f031c0a3cc0 msgr2=0x7f031c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:01.731 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f031c0a3cc0 0x7f031c0a6180 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b5c0 tx=0x7f032c003860 comp rx=0 tx=0).stop 2024-04-28T12:45:01.731 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330105020 msgr2=0x7f0330075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:01.731 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330105020 0x7f0330075d50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f032800b970 tx=0x7f032800bc80 comp rx=0 tx=0).stop 2024-04-28T12:45:01.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 shutdown_connections 2024-04-28T12:45:01.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f031c0a3cc0 0x7f031c0a6180 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:01.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0330105020 0x7f0330075d50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:01.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0330103480 0x7f0330077170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:01.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 --2- 172.21.15.171:0/2281187995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0330104680 0x7f0330075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:01.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 >> 172.21.15.171:0/2281187995 conn(0x7f03300fea20 msgr2=0x7f0330107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:45:01.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 shutdown_connections 2024-04-28T12:45:01.732 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:45:01.735+0000 7f03387a1700 1 -- 172.21.15.171:0/2281187995 wait complete. 2024-04-28T12:45:02.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:02 smithi106 ceph-mon[28081]: from='osd.5 [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-28T12:45:02.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:02 smithi106 ceph-mon[28081]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-28T12:45:02.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:02.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:02.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:02.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:02 smithi171 ceph-mon[63127]: from='osd.5 [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-28T12:45:02.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:02 smithi171 ceph-mon[63127]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-28T12:45:02.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:02.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:02.359 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:02.384 DEBUG:teuthology.orchestra.run.smithi171:osd.5> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.5.service 2024-04-28T12:45:02.386 INFO:tasks.cephadm:Deploying osd.6 on smithi106 with /dev/vg_nvme/lv_2... 2024-04-28T12:45:02.386 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-28T12:45:02.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:02 smithi129 ceph-mon[62418]: from='osd.5 [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-28T12:45:02.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:02 smithi129 ceph-mon[62418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-28T12:45:02.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:02.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:02.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:02.826 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:45:03.292 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:03 smithi106 ceph-mon[28081]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:03.293 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:03 smithi106 ceph-mon[28081]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-28T12:45:03.293 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:03 smithi106 ceph-mon[28081]: from='osd.5 [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:45:03.293 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:03 smithi106 ceph-mon[28081]: osdmap e36: 6 total, 5 up, 6 in 2024-04-28T12:45:03.293 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:03 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:45:03.293 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:03 smithi106 ceph-mon[28081]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:45:03.293 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:03 smithi106 ceph-mon[28081]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:45:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:03 smithi171 ceph-mon[63127]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:03 smithi171 ceph-mon[63127]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-28T12:45:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:03 smithi171 ceph-mon[63127]: from='osd.5 [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:45:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:03 smithi171 ceph-mon[63127]: osdmap e36: 6 total, 5 up, 6 in 2024-04-28T12:45:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:03 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:45:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:03 smithi171 ceph-mon[63127]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:45:03.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:03 smithi171 ceph-mon[63127]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:45:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:03 smithi129 ceph-mon[62418]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:03 smithi129 ceph-mon[62418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-28T12:45:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:03 smithi129 ceph-mon[62418]: from='osd.5 [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:45:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:03 smithi129 ceph-mon[62418]: osdmap e36: 6 total, 5 up, 6 in 2024-04-28T12:45:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:03 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:45:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:03 smithi129 ceph-mon[62418]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:45:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:03 smithi129 ceph-mon[62418]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:45:03.941 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 12:45:03 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T12:45:03.759+0000 7f4454b72700 -1 osd.5 0 waiting for initial osdmap 2024-04-28T12:45:03.941 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 12:45:03 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T12:45:03.765+0000 7f444c95d700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:45:04.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:04 smithi171 ceph-mon[63127]: osdmap e37: 6 total, 5 up, 6 in 2024-04-28T12:45:04.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:45:04.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:04 smithi171 ceph-mon[63127]: from='osd.5 ' entity='osd.5' 2024-04-28T12:45:04.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:04 smithi171 ceph-mon[63127]: osd.5 [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] boot 2024-04-28T12:45:04.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:04 smithi171 ceph-mon[63127]: osdmap e38: 6 total, 6 up, 6 in 2024-04-28T12:45:04.219 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:45:04.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:04 smithi106 ceph-mon[28081]: osdmap e37: 6 total, 5 up, 6 in 2024-04-28T12:45:04.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:45:04.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:04 smithi106 ceph-mon[28081]: from='osd.5 ' entity='osd.5' 2024-04-28T12:45:04.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:04 smithi106 ceph-mon[28081]: osd.5 [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] boot 2024-04-28T12:45:04.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:04 smithi106 ceph-mon[28081]: osdmap e38: 6 total, 6 up, 6 in 2024-04-28T12:45:04.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:45:04.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:04 smithi129 ceph-mon[62418]: osdmap e37: 6 total, 5 up, 6 in 2024-04-28T12:45:04.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:04 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:45:04.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:04 smithi129 ceph-mon[62418]: from='osd.5 ' entity='osd.5' 2024-04-28T12:45:04.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:04 smithi129 ceph-mon[62418]: osd.5 [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] boot 2024-04-28T12:45:04.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:04 smithi129 ceph-mon[62418]: osdmap e38: 6 total, 6 up, 6 in 2024-04-28T12:45:04.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:04 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-28T12:45:05.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:05 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:45:05.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:05 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:45:05.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:05 smithi106 ceph-mon[28081]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:05.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:05 smithi106 ceph-mon[28081]: osdmap e39: 6 total, 6 up, 6 in 2024-04-28T12:45:05.398 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:05 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:45:05.398 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:05 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:45:05.398 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:05 smithi171 ceph-mon[63127]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:05.398 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:05 smithi171 ceph-mon[63127]: osdmap e39: 6 total, 6 up, 6 in 2024-04-28T12:45:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:05 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:45:05.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:05 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:45:05.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:05 smithi129 ceph-mon[62418]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-28T12:45:05.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:05 smithi129 ceph-mon[62418]: osdmap e39: 6 total, 6 up, 6 in 2024-04-28T12:45:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-28T12:45:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-28T12:45:05.762 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-28T12:45:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-28T12:45:05.762 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0356791 s, 294 MB/s 2024-04-28T12:45:05.762 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-28T12:45:06.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_2 2024-04-28T12:45:06.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi171 2024-04-28T12:45:06.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:06.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:06.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-28T12:45:06.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:06.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:06.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:06.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:06 smithi171 ceph-mon[63127]: osdmap e40: 6 total, 6 up, 6 in 2024-04-28T12:45:06.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi171 2024-04-28T12:45:06.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:06.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-28T12:45:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:06.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:06 smithi129 ceph-mon[62418]: osdmap e40: 6 total, 6 up, 6 in 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi171 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi171 to 8544M 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: pgmap v154: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:06.768 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:06 smithi106 ceph-mon[28081]: osdmap e40: 6 total, 6 up, 6 in 2024-04-28T12:45:07.131 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:45:08.299 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:08 smithi106 ceph-mon[28081]: osdmap e41: 6 total, 6 up, 6 in 2024-04-28T12:45:08.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:08 smithi171 ceph-mon[63127]: osdmap e41: 6 total, 6 up, 6 in 2024-04-28T12:45:08.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:08 smithi129 ceph-mon[62418]: osdmap e41: 6 total, 6 up, 6 in 2024-04-28T12:45:09.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.158+0000 7f66d2800700 1 -- 172.21.15.106:0/2688031668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc107040 msgr2=0x7f66cc1074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.158+0000 7f66d2800700 1 --2- 172.21.15.106:0/2688031668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc107040 0x7f66cc1074c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66c400b0a0 tx=0x7f66c400b3b0 comp rx=0 tx=0).stop 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.158+0000 7f66d2800700 1 -- 172.21.15.106:0/2688031668 shutdown_connections 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.158+0000 7f66d2800700 1 --2- 172.21.15.106:0/2688031668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc107040 0x7f66cc1074c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.158+0000 7f66d2800700 1 --2- 172.21.15.106:0/2688031668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66cc101d00 0x7f66cc104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.158+0000 7f66d2800700 1 --2- 172.21.15.106:0/2688031668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66cc104670 0x7f66cc106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.158+0000 7f66d2800700 1 -- 172.21.15.106:0/2688031668 >> 172.21.15.106:0/2688031668 conn(0x7f66cc0fd920 msgr2=0x7f66cc0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.158+0000 7f66d2800700 1 -- 172.21.15.106:0/2688031668 shutdown_connections 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.159+0000 7f66d2800700 1 -- 172.21.15.106:0/2688031668 wait complete. 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.159+0000 7f66d2800700 1 Processor -- start 2024-04-28T12:45:09.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66d2800700 1 -- start start 2024-04-28T12:45:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66d2800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66cc101d00 0x7f66cc19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:45:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66d2800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc104670 0x7f66cc1a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:45:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66d2800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66cc107040 0x7f66cc1a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:45:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66d2800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc115490 con 0x7f66cc104670 2024-04-28T12:45:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66d2800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc115310 con 0x7f66cc101d00 2024-04-28T12:45:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66d2800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc115610 con 0x7f66cc107040 2024-04-28T12:45:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66caa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc104670 0x7f66cc1a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:45:09.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66caa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc104670 0x7f66cc1a0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34718/0 (socket says 172.21.15.106:34718) 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66caa00700 1 -- 172.21.15.106:0/1126892078 learned_addr learned my addr 172.21.15.106:0/1126892078 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66cb400700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66cc101d00 0x7f66cc19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.160+0000 7f66cbe00700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66cc107040 0x7f66cc1a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66caa00700 1 -- 172.21.15.106:0/1126892078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66cc107040 msgr2=0x7f66cc1a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66caa00700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66cc107040 0x7f66cc1a70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66caa00700 1 -- 172.21.15.106:0/1126892078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66cc101d00 msgr2=0x7f66cc19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66caa00700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66cc101d00 0x7f66cc19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66caa00700 1 -- 172.21.15.106:0/1126892078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66cc1a7760 con 0x7f66cc104670 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66caa00700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc104670 0x7f66cc1a0090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f66b800eb60 tx=0x7f66b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66c3e00700 1 -- 172.21.15.106:0/1126892078 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b800cbd0 con 0x7f66cc104670 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66c3e00700 1 -- 172.21.15.106:0/1126892078 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b80044d0 con 0x7f66cc104670 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66cc1a7a50 con 0x7f66cc104670 2024-04-28T12:45:09.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.161+0000 7f66c3e00700 1 -- 172.21.15.106:0/1126892078 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8010430 con 0x7f66cc104670 2024-04-28T12:45:09.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.162+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66cc1a7fe0 con 0x7f66cc104670 2024-04-28T12:45:09.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.163+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66cc06a110 con 0x7f66cc104670 2024-04-28T12:45:09.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.165+0000 7f66c3e00700 1 -- 172.21.15.106:0/1126892078 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f66b8003e70 con 0x7f66cc104670 2024-04-28T12:45:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.166+0000 7f66c3e00700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f66b00a3cc0 0x7f66b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:45:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.167+0000 7f66c3e00700 1 -- 172.21.15.106:0/1126892078 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f66b80cf6a0 con 0x7f66cc104670 2024-04-28T12:45:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.167+0000 7f66cb400700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f66b00a3cc0 0x7f66b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:45:09.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.167+0000 7f66cb400700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f66b00a3cc0 0x7f66b00a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009c60 tx=0x7f66bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:45:09.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.171+0000 7f66c3e00700 1 -- 172.21.15.106:0/1126892078 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f66b809cf40 con 0x7f66cc104670 2024-04-28T12:45:09.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:09 smithi106 ceph-mon[28081]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:09.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:09 smithi106 ceph-mon[28081]: osdmap e42: 6 total, 6 up, 6 in 2024-04-28T12:45:09.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:09.390+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f66cc063860 con 0x7f66b00a3cc0 2024-04-28T12:45:09.396 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:09 smithi171 ceph-mon[63127]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:09.396 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:09 smithi171 ceph-mon[63127]: osdmap e42: 6 total, 6 up, 6 in 2024-04-28T12:45:09.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:09 smithi129 ceph-mon[62418]: pgmap v157: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:09.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:09 smithi129 ceph-mon[62418]: osdmap e42: 6 total, 6 up, 6 in 2024-04-28T12:45:10.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:45:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:45:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:10.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:45:10.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:45:10.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:45:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:45:10.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:11 smithi106 ceph-mon[28081]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:45:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:11 smithi106 ceph-mon[28081]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:11 smithi171 ceph-mon[63127]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:45:11.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:11 smithi171 ceph-mon[63127]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:11.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:11 smithi129 ceph-mon[62418]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:45:11.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:11 smithi129 ceph-mon[62418]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:13.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:13 smithi106 ceph-mon[28081]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-28T12:45:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:13 smithi171 ceph-mon[63127]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-28T12:45:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:13 smithi129 ceph-mon[62418]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-28T12:45:14.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:14 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2062162927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17da728b-3acb-48d3-9fda-4485aa4b3757"}]: dispatch 2024-04-28T12:45:14.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:14 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17da728b-3acb-48d3-9fda-4485aa4b3757"}]: dispatch 2024-04-28T12:45:14.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:14 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17da728b-3acb-48d3-9fda-4485aa4b3757"}]': finished 2024-04-28T12:45:14.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:14 smithi106 ceph-mon[28081]: osdmap e43: 7 total, 6 up, 7 in 2024-04-28T12:45:14.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:14 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2062162927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17da728b-3acb-48d3-9fda-4485aa4b3757"}]: dispatch 2024-04-28T12:45:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:14 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17da728b-3acb-48d3-9fda-4485aa4b3757"}]: dispatch 2024-04-28T12:45:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:14 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17da728b-3acb-48d3-9fda-4485aa4b3757"}]': finished 2024-04-28T12:45:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:14 smithi171 ceph-mon[63127]: osdmap e43: 7 total, 6 up, 7 in 2024-04-28T12:45:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:14.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:14 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2062162927' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17da728b-3acb-48d3-9fda-4485aa4b3757"}]: dispatch 2024-04-28T12:45:14.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:14 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17da728b-3acb-48d3-9fda-4485aa4b3757"}]: dispatch 2024-04-28T12:45:14.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:14 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17da728b-3acb-48d3-9fda-4485aa4b3757"}]': finished 2024-04-28T12:45:14.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:14 smithi129 ceph-mon[62418]: osdmap e43: 7 total, 6 up, 7 in 2024-04-28T12:45:14.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:15.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:15 smithi106 ceph-mon[28081]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-28T12:45:15.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:15 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3196567137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:45:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:15 smithi171 ceph-mon[63127]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-28T12:45:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:15 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3196567137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:45:15.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:15 smithi129 ceph-mon[62418]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-28T12:45:15.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:15 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3196567137' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:45:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:17 smithi171 ceph-mon[63127]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-28T12:45:17.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:17 smithi129 ceph-mon[62418]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-28T12:45:17.543 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:17 smithi106 ceph-mon[28081]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-28T12:45:19.352 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:19 smithi106 ceph-mon[28081]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-28T12:45:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:19 smithi171 ceph-mon[63127]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-28T12:45:19.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:19 smithi129 ceph-mon[62418]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-28T12:45:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:21 smithi171 ceph-mon[63127]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-28T12:45:21.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:21 smithi129 ceph-mon[62418]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-28T12:45:21.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:21 smithi106 ceph-mon[28081]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-28T12:45:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:23 smithi171 ceph-mon[63127]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:23 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-28T12:45:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:23 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:23.497 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:23 smithi106 ceph-mon[28081]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:23.497 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:23 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-28T12:45:23.497 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:23 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:23.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:23 smithi129 ceph-mon[62418]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:23.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:23 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-28T12:45:23.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:23 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:24.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:24 smithi171 ceph-mon[63127]: Deploying daemon osd.6 on smithi106 2024-04-28T12:45:24.508 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:24 smithi106 ceph-mon[28081]: Deploying daemon osd.6 on smithi106 2024-04-28T12:45:24.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:24 smithi129 ceph-mon[62418]: Deploying daemon osd.6 on smithi106 2024-04-28T12:45:25.443 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:25 smithi106 ceph-mon[28081]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:25 smithi171 ceph-mon[63127]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:25.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:25 smithi129 ceph-mon[62418]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:27.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:27 smithi106 ceph-mon[28081]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:27 smithi171 ceph-mon[63127]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:27 smithi129 ceph-mon[62418]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:29 smithi171 ceph-mon[63127]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:29 smithi129 ceph-mon[62418]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:29.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:29 smithi106 ceph-mon[28081]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:31.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:31 smithi171 ceph-mon[63127]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:31.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:31 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:31.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:31 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:31.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:31 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:31 smithi129 ceph-mon[62418]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:31 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:31 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:31 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:31.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:31 smithi106 ceph-mon[28081]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:31.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:31 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:31.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:31 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:31.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:31 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:33.450 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:33 smithi106 ceph-mon[28081]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:33 smithi171 ceph-mon[63127]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:33 smithi129 ceph-mon[62418]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:35 smithi171 ceph-mon[63127]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:35 smithi171 ceph-mon[63127]: from='osd.6 [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-28T12:45:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:35 smithi171 ceph-mon[63127]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-28T12:45:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:35 smithi129 ceph-mon[62418]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:35 smithi129 ceph-mon[62418]: from='osd.6 [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-28T12:45:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:35 smithi129 ceph-mon[62418]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-28T12:45:35.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:35 smithi106 ceph-mon[28081]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:35.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:35.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:35.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:35 smithi106 ceph-mon[28081]: from='osd.6 [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-28T12:45:35.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:35 smithi106 ceph-mon[28081]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-28T12:45:35.788 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 6 on host 'smithi106' 2024-04-28T12:45:35.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.791+0000 7f66c3e00700 1 -- 172.21.15.106:0/1126892078 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f66cc063860 con 0x7f66b00a3cc0 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f66b00a3cc0 msgr2=0x7f66b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f66b00a3cc0 0x7f66b00a6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f66bc009c60 tx=0x7f66bc009260 comp rx=0 tx=0).stop 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc104670 msgr2=0x7f66cc1a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc104670 0x7f66cc1a0090 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f66b800eb60 tx=0x7f66b800ee70 comp rx=0 tx=0).stop 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 shutdown_connections 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f66b00a3cc0 0x7f66b00a6180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66cc107040 0x7f66cc1a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66cc101d00 0x7f66cc19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 --2- 172.21.15.106:0/1126892078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66cc104670 0x7f66cc1a0090 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 >> 172.21.15.106:0/1126892078 conn(0x7f66cc0fd920 msgr2=0x7f66cc102680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 shutdown_connections 2024-04-28T12:45:35.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:45:35.796+0000 7f66d2800700 1 -- 172.21.15.106:0/1126892078 wait complete. 2024-04-28T12:45:36.591 DEBUG:teuthology.orchestra.run.smithi106:osd.6> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.6.service 2024-04-28T12:45:36.595 INFO:tasks.cephadm:Deploying osd.7 on smithi129 with /dev/vg_nvme/lv_2... 2024-04-28T12:45:36.595 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-28T12:45:36.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:36 smithi106 ceph-mon[28081]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-28T12:45:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:36 smithi106 ceph-mon[28081]: from='osd.6 [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:45:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:36 smithi106 ceph-mon[28081]: osdmap e44: 7 total, 6 up, 7 in 2024-04-28T12:45:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:36 smithi106 ceph-mon[28081]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:45:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:36.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:36 smithi106 ceph-mon[28081]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:36.836 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.b/config 2024-04-28T12:45:36.958 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:36 smithi129 ceph-mon[62418]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-28T12:45:36.958 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:36 smithi129 ceph-mon[62418]: from='osd.6 [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:45:36.958 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:36 smithi129 ceph-mon[62418]: osdmap e44: 7 total, 6 up, 7 in 2024-04-28T12:45:36.958 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:36.958 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:36 smithi129 ceph-mon[62418]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:45:36.958 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:36.958 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:36.958 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:36 smithi129 ceph-mon[62418]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:36.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:36 smithi171 ceph-mon[63127]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-28T12:45:36.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:36 smithi171 ceph-mon[63127]: from='osd.6 [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:45:36.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:36 smithi171 ceph-mon[63127]: osdmap e44: 7 total, 6 up, 7 in 2024-04-28T12:45:36.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:36.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:36 smithi171 ceph-mon[63127]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:45:36.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:36.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:36.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:36 smithi171 ceph-mon[63127]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-28T12:45:37.944 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:37 smithi129 ceph-mon[62418]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:45:37.945 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:37 smithi129 ceph-mon[62418]: osdmap e45: 7 total, 6 up, 7 in 2024-04-28T12:45:37.945 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:37 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:37.945 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:37 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:37.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:37 smithi171 ceph-mon[63127]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:45:37.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:37 smithi171 ceph-mon[63127]: osdmap e45: 7 total, 6 up, 7 in 2024-04-28T12:45:37.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:37 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:37.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:37 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:38.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:37 smithi106 ceph-mon[28081]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:45:38.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:37 smithi106 ceph-mon[28081]: osdmap e45: 7 total, 6 up, 7 in 2024-04-28T12:45:38.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:37 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:38.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:37 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:38.740 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:38 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:45:38.740 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:38 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:45:38.740 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:38 smithi106 ceph-mon[28081]: osd.6 [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] boot 2024-04-28T12:45:38.740 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:38 smithi106 ceph-mon[28081]: osdmap e46: 7 total, 7 up, 7 in 2024-04-28T12:45:38.740 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:38 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:38.740 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:38 smithi106 ceph-mon[28081]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:38.823 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:38 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:45:38.823 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:38 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:45:38.823 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:38 smithi129 ceph-mon[62418]: osd.6 [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] boot 2024-04-28T12:45:38.823 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:38 smithi129 ceph-mon[62418]: osdmap e46: 7 total, 7 up, 7 in 2024-04-28T12:45:38.823 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:38 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:38.824 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:38 smithi129 ceph-mon[62418]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:38.824 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-28T12:45:38.824 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-28T12:45:38.824 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-28T12:45:38.824 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-28T12:45:38.824 INFO:teuthology.orchestra.run.smithi129.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011303 s, 928 MB/s 2024-04-28T12:45:38.824 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-28T12:45:38.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:38 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:45:38.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:38 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:45:38.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:38 smithi171 ceph-mon[63127]: osd.6 [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] boot 2024-04-28T12:45:38.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:38 smithi171 ceph-mon[63127]: osdmap e46: 7 total, 7 up, 7 in 2024-04-28T12:45:38.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:38 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-28T12:45:38.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:38 smithi171 ceph-mon[63127]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:39.488 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_2 2024-04-28T12:45:39.729 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.b/config 2024-04-28T12:45:39.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:39 smithi129 ceph-mon[62418]: osdmap e47: 7 total, 7 up, 7 in 2024-04-28T12:45:39.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:39 smithi171 ceph-mon[63127]: osdmap e47: 7 total, 7 up, 7 in 2024-04-28T12:45:40.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:39 smithi106 ceph-mon[28081]: osdmap e47: 7 total, 7 up, 7 in 2024-04-28T12:45:40.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:40 smithi129 ceph-mon[62418]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:40.773 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:40 smithi106 ceph-mon[28081]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:40.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:40 smithi171 ceph-mon[63127]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:41.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.352+0000 7f22fa4b5700 1 -- 172.21.15.129:0/2616691678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 msgr2=0x7f22f41038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:41.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.352+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/2616691678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 0x7f22f41038a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f22e40099c0 tx=0x7f22e4009cd0 comp rx=0 tx=0).stop 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.352+0000 7f22fa4b5700 1 -- 172.21.15.129:0/2616691678 shutdown_connections 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.352+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/2616691678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f4105020 0x7f22f410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.352+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/2616691678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 0x7f22f41038a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.352+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/2616691678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22f4104680 0x7f22f4104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.352+0000 7f22fa4b5700 1 -- 172.21.15.129:0/2616691678 >> 172.21.15.129:0/2616691678 conn(0x7f22f40fea20 msgr2=0x7f22f4100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.352+0000 7f22fa4b5700 1 -- 172.21.15.129:0/2616691678 shutdown_connections 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.353+0000 7f22fa4b5700 1 -- 172.21.15.129:0/2616691678 wait complete. 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.353+0000 7f22fa4b5700 1 Processor -- start 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.353+0000 7f22fa4b5700 1 -- start start 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.353+0000 7f22fa4b5700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 0x7f22f40771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.353+0000 7f22fa4b5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22f4104680 0x7f22f4075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.353+0000 7f22fa4b5700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f4105020 0x7f22f4075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:45:41.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.353+0000 7f22fa4b5700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22f410ef80 con 0x7f22f4105020 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.353+0000 7f22fa4b5700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22f410ee00 con 0x7f22f4103480 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.353+0000 7f22fa4b5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f22f410f100 con 0x7f22f4104680 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f3fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 0x7f22f40771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f3fff700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 0x7f22f40771c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41602/0 (socket says 172.21.15.129:41602) 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f37fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22f4104680 0x7f22f4075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f3fff700 1 -- 172.21.15.129:0/1995101426 learned_addr learned my addr 172.21.15.129:0/1995101426 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f3fff700 1 -- 172.21.15.129:0/1995101426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22f4104680 msgr2=0x7f22f4075810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f3fff700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22f4104680 0x7f22f4075810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f3fff700 1 -- 172.21.15.129:0/1995101426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f4105020 msgr2=0x7f22f4075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f3fff700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f4105020 0x7f22f4075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f3fff700 1 -- 172.21.15.129:0/1995101426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f4076600 con 0x7f22f4103480 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f37fe700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22f4104680 0x7f22f4075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f3fff700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 0x7f22f40771c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009f50 tx=0x7f22e40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f17fa700 1 -- 172.21.15.129:0/1995101426 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4027070 con 0x7f22f4103480 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f41a3780 con 0x7f22f4103480 2024-04-28T12:45:41.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f17fa700 1 -- 172.21.15.129:0/1995101426 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22e4022470 con 0x7f22f4103480 2024-04-28T12:45:41.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22f17fa700 1 -- 172.21.15.129:0/1995101426 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e400f6c0 con 0x7f22f4103480 2024-04-28T12:45:41.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.354+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f41a3c10 con 0x7f22f4103480 2024-04-28T12:45:41.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.355+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f406a110 con 0x7f22f4103480 2024-04-28T12:45:41.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.356+0000 7f22f17fa700 1 -- 172.21.15.129:0/1995101426 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f22e402a070 con 0x7f22f4103480 2024-04-28T12:45:41.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.357+0000 7f22f17fa700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22dc0a3c40 0x7f22dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:45:41.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.357+0000 7f22f17fa700 1 -- 172.21.15.129:0/1995101426 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f22e40d0680 con 0x7f22f4103480 2024-04-28T12:45:41.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.357+0000 7f22f37fe700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22dc0a3c40 0x7f22dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:45:41.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.358+0000 7f22f37fe700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22dc0a3c40 0x7f22dc0a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22e8005950 tx=0x7f22e800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:45:41.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.362+0000 7f22f17fa700 1 -- 172.21.15.129:0/1995101426 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f22e4013070 con 0x7f22f4103480 2024-04-28T12:45:41.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:45:41.573+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f22f4063860 con 0x7f22dc0a3c40 2024-04-28T12:45:42.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi106 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:45:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:45:42.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:42.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:42.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:42.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:42.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:42 smithi129 ceph-mon[62418]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:42.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi106 2024-04-28T12:45:42.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:42.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:42 smithi171 ceph-mon[63127]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi106 2024-04-28T12:45:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:45:42.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='client.24290 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:42.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:42 smithi106 ceph-mon[28081]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:45 smithi106 ceph-mon[28081]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:45 smithi106 ceph-mon[28081]: from='client.? 172.21.15.129:0/558850944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d679bae-c3a9-4c6b-a604-934ae327ca2f"}]: dispatch 2024-04-28T12:45:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:45 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d679bae-c3a9-4c6b-a604-934ae327ca2f"}]: dispatch 2024-04-28T12:45:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:45 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d679bae-c3a9-4c6b-a604-934ae327ca2f"}]': finished 2024-04-28T12:45:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:45 smithi106 ceph-mon[28081]: osdmap e48: 8 total, 7 up, 8 in 2024-04-28T12:45:45.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:45 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:45:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:45 smithi171 ceph-mon[63127]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:45 smithi171 ceph-mon[63127]: from='client.? 172.21.15.129:0/558850944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d679bae-c3a9-4c6b-a604-934ae327ca2f"}]: dispatch 2024-04-28T12:45:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:45 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d679bae-c3a9-4c6b-a604-934ae327ca2f"}]: dispatch 2024-04-28T12:45:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:45 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d679bae-c3a9-4c6b-a604-934ae327ca2f"}]': finished 2024-04-28T12:45:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:45 smithi171 ceph-mon[63127]: osdmap e48: 8 total, 7 up, 8 in 2024-04-28T12:45:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:45 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:45:45.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:45 smithi129 ceph-mon[62418]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:45.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:45 smithi129 ceph-mon[62418]: from='client.? 172.21.15.129:0/558850944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d679bae-c3a9-4c6b-a604-934ae327ca2f"}]: dispatch 2024-04-28T12:45:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:45 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d679bae-c3a9-4c6b-a604-934ae327ca2f"}]: dispatch 2024-04-28T12:45:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:45 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d679bae-c3a9-4c6b-a604-934ae327ca2f"}]': finished 2024-04-28T12:45:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:45 smithi129 ceph-mon[62418]: osdmap e48: 8 total, 7 up, 8 in 2024-04-28T12:45:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:45 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:45:46.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:46 smithi106 ceph-mon[28081]: from='client.? 172.21.15.129:0/3097036718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:45:46.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:46 smithi171 ceph-mon[63127]: from='client.? 172.21.15.129:0/3097036718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:45:46.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:46 smithi129 ceph-mon[62418]: from='client.? 172.21.15.129:0/3097036718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:45:47.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:47 smithi106 ceph-mon[28081]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:47.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:47 smithi171 ceph-mon[63127]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:47 smithi129 ceph-mon[62418]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:49 smithi106 ceph-mon[28081]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:49.383 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:49 smithi129 ceph-mon[62418]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:49 smithi171 ceph-mon[63127]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:51.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:51 smithi106 ceph-mon[28081]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:51.476 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:51 smithi129 ceph-mon[62418]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:51.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:51 smithi171 ceph-mon[63127]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:52.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-28T12:45:52.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-28T12:45:52.383 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-28T12:45:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:45:53.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:53 smithi129 ceph-mon[62418]: Deploying daemon osd.7 on smithi129 2024-04-28T12:45:53.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:53 smithi129 ceph-mon[62418]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:53.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:53 smithi106 ceph-mon[28081]: Deploying daemon osd.7 on smithi129 2024-04-28T12:45:53.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:53 smithi106 ceph-mon[28081]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:53.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:53 smithi171 ceph-mon[63127]: Deploying daemon osd.7 on smithi129 2024-04-28T12:45:53.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:53 smithi171 ceph-mon[63127]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:55.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:55 smithi129 ceph-mon[62418]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:55 smithi171 ceph-mon[63127]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:55 smithi106 ceph-mon[28081]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:57.293 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:57 smithi129 ceph-mon[62418]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:57 smithi171 ceph-mon[63127]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:57.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:57 smithi106 ceph-mon[28081]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:58.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:58.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:58.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:58.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:45:58 smithi171 ceph-mon[63127]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:59.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:59.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:59.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:59.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:45:58 smithi129 ceph-mon[62418]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:45:59.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:59.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:45:59.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:45:59.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:45:58 smithi106 ceph-mon[28081]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:01.162 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 7 on host 'smithi129' 2024-04-28T12:46:01.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.165+0000 7f22f17fa700 1 -- 172.21.15.129:0/1995101426 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f22f4063860 con 0x7f22dc0a3c40 2024-04-28T12:46:01.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22dc0a3c40 msgr2=0x7f22dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:01.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22dc0a3c40 0x7f22dc0a6100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22e8005950 tx=0x7f22e800a300 comp rx=0 tx=0).stop 2024-04-28T12:46:01.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 msgr2=0x7f22f40771c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:01.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 0x7f22f40771c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009f50 tx=0x7f22e40047b0 comp rx=0 tx=0).stop 2024-04-28T12:46:01.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 shutdown_connections 2024-04-28T12:46:01.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22f4105020 0x7f22f4075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:01.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f22f4103480 0x7f22f40771c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:01.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22dc0a3c40 0x7f22dc0a6100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:01.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 --2- 172.21.15.129:0/1995101426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22f4104680 0x7f22f4075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:01.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 >> 172.21.15.129:0/1995101426 conn(0x7f22f40fea20 msgr2=0x7f22f4107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:46:01.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 shutdown_connections 2024-04-28T12:46:01.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:46:01.168+0000 7f22fa4b5700 1 -- 172.21.15.129:0/1995101426 wait complete. 2024-04-28T12:46:01.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:01 smithi106 ceph-mon[28081]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:01.348 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:01 smithi129 ceph-mon[62418]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:01 smithi171 ceph-mon[63127]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:02.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:02.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:02.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:02 smithi129 ceph-mon[62418]: from='osd.7 [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-28T12:46:02.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:02 smithi129 ceph-mon[62418]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-28T12:46:02.317 DEBUG:teuthology.orchestra.run.smithi129:osd.7> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.7.service 2024-04-28T12:46:02.319 INFO:tasks.cephadm:Deploying osd.8 on smithi171 with /dev/vg_nvme/lv_2... 2024-04-28T12:46:02.319 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-28T12:46:02.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:02.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:02.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:02 smithi171 ceph-mon[63127]: from='osd.7 [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-28T12:46:02.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:02 smithi171 ceph-mon[63127]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-28T12:46:02.568 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:46:02.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:02.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:02.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:02 smithi106 ceph-mon[28081]: from='osd.7 [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-28T12:46:02.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:02 smithi106 ceph-mon[28081]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-28T12:46:03.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-mon[62418]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:03.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-mon[62418]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-28T12:46:03.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-mon[62418]: from='osd.7 [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:46:03.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-mon[62418]: osdmap e49: 8 total, 7 up, 8 in 2024-04-28T12:46:03.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:03.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-mon[62418]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:46:03.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:03.315 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:03.315 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T12:46:03.135+0000 7f3f24f3a700 -1 osd.7 0 waiting for initial osdmap 2024-04-28T12:46:03.316 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 12:46:03 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T12:46:03.144+0000 7f3f20d2d700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:46:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:03 smithi171 ceph-mon[63127]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:03 smithi171 ceph-mon[63127]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-28T12:46:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:03 smithi171 ceph-mon[63127]: from='osd.7 [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:46:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:03 smithi171 ceph-mon[63127]: osdmap e49: 8 total, 7 up, 8 in 2024-04-28T12:46:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:03 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:03 smithi171 ceph-mon[63127]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:46:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:03 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:03 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:03.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:03 smithi106 ceph-mon[28081]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:03.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:03 smithi106 ceph-mon[28081]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-28T12:46:03.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:03 smithi106 ceph-mon[28081]: from='osd.7 [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:46:03.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:03 smithi106 ceph-mon[28081]: osdmap e49: 8 total, 7 up, 8 in 2024-04-28T12:46:03.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:03 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:03.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:03 smithi106 ceph-mon[28081]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:46:03.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:03 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:03.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:03 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:04.441 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:04 smithi129 ceph-mon[62418]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:46:04.441 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:04 smithi129 ceph-mon[62418]: osdmap e50: 8 total, 7 up, 8 in 2024-04-28T12:46:04.441 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:04 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:04.441 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:04 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:04.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:04 smithi171 ceph-mon[63127]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:46:04.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:04 smithi171 ceph-mon[63127]: osdmap e50: 8 total, 7 up, 8 in 2024-04-28T12:46:04.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:04.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:04.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:04 smithi106 ceph-mon[28081]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:46:04.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:04 smithi106 ceph-mon[28081]: osdmap e50: 8 total, 7 up, 8 in 2024-04-28T12:46:04.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:04.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:04.619 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-28T12:46:04.619 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-28T12:46:04.619 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-28T12:46:04.620 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-28T12:46:04.620 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113057 s, 927 MB/s 2024-04-28T12:46:04.620 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-28T12:46:05.329 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2024-04-28T12:46:05.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:05 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:46:05.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:05 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:46:05.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:05 smithi171 ceph-mon[63127]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:05.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:05 smithi171 ceph-mon[63127]: osd.7 [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] boot 2024-04-28T12:46:05.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:05 smithi171 ceph-mon[63127]: osdmap e51: 8 total, 8 up, 8 in 2024-04-28T12:46:05.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:05 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:05 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:46:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:05 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:46:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:05 smithi129 ceph-mon[62418]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:05 smithi129 ceph-mon[62418]: osd.7 [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] boot 2024-04-28T12:46:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:05 smithi129 ceph-mon[62418]: osdmap e51: 8 total, 8 up, 8 in 2024-04-28T12:46:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:05 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:05.573 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:46:05.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:05 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:46:05.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:05 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:46:05.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:05 smithi106 ceph-mon[28081]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-28T12:46:05.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:05 smithi106 ceph-mon[28081]: osd.7 [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] boot 2024-04-28T12:46:05.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:05 smithi106 ceph-mon[28081]: osdmap e51: 8 total, 8 up, 8 in 2024-04-28T12:46:05.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:05 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-28T12:46:06.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: osdmap e52: 8 total, 8 up, 8 in 2024-04-28T12:46:06.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:06.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:06.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:06.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:06.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:06.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:06 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: osdmap e52: 8 total, 8 up, 8 in 2024-04-28T12:46:06.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:06.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:06.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:06.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:06.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:06.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:06 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: osdmap e52: 8 total, 8 up, 8 in 2024-04-28T12:46:06.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:06.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:06.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:06.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:06.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:06.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:06.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:06 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:07.307 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.309+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/2223495726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01091d0 msgr2=0x7f2fa010b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:07.307 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.309+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/2223495726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01091d0 0x7f2fa010b5e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800b0a0 tx=0x7f2f9800b3b0 comp rx=0 tx=0).stop 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/2223495726 shutdown_connections 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/2223495726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01091d0 0x7f2fa010b5e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/2223495726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fa0103f70 0x7f2fa0106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/2223495726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fa01068a0 0x7f2fa0108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/2223495726 >> 172.21.15.171:0/2223495726 conn(0x7f2fa00fd940 msgr2=0x7f2fa00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/2223495726 shutdown_connections 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/2223495726 wait complete. 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 Processor -- start 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 -- start start 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fa0103f70 0x7f2fa01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01068a0 0x7f2fa0199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fa01091d0 0x7f2fa01a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa010efe0 con 0x7f2fa0103f70 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa010ee60 con 0x7f2fa01091d0 2024-04-28T12:46:07.308 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.310+0000 7f2fa7a4b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa010f160 con 0x7f2fa01068a0 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa4fe6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01068a0 0x7f2fa0199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa4fe6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01068a0 0x7f2fa0199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:47346/0 (socket says 172.21.15.171:47346) 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa4fe6700 1 -- 172.21.15.171:0/1953946821 learned_addr learned my addr 172.21.15.171:0/1953946821 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa5fe8700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fa01091d0 0x7f2fa01a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa4fe6700 1 -- 172.21.15.171:0/1953946821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fa01091d0 msgr2=0x7f2fa01a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa4fe6700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fa01091d0 0x7f2fa01a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa4fe6700 1 -- 172.21.15.171:0/1953946821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fa0103f70 msgr2=0x7f2fa01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa4fe6700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fa0103f70 0x7f2fa01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa4fe6700 1 -- 172.21.15.171:0/1953946821 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa01a1310 con 0x7f2fa01068a0 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa4fe6700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01068a0 0x7f2fa0199bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000b560 tx=0x7f2f9000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2f967fc700 1 -- 172.21.15.171:0/1953946821 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90015020 con 0x7f2fa01068a0 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2f967fc700 1 -- 172.21.15.171:0/1953946821 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f90011ce0 con 0x7f2fa01068a0 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa01a1600 con 0x7f2fa01068a0 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2f967fc700 1 -- 172.21.15.171:0/1953946821 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90010d20 con 0x7f2fa01068a0 2024-04-28T12:46:07.309 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.311+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa01a1bc0 con 0x7f2fa01068a0 2024-04-28T12:46:07.310 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.312+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa006a110 con 0x7f2fa01068a0 2024-04-28T12:46:07.310 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.313+0000 7f2f967fc700 1 -- 172.21.15.171:0/1953946821 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2f90018020 con 0x7f2fa01068a0 2024-04-28T12:46:07.310 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.314+0000 7f2f967fc700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2f8c0a3c40 0x7f2f8c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:46:07.310 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.314+0000 7f2f967fc700 1 -- 172.21.15.171:0/1953946821 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5430+0+0 (secure 0 0 0) 0x7f2f900cf5b0 con 0x7f2fa01068a0 2024-04-28T12:46:07.311 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.314+0000 7f2fa57e7700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2f8c0a3c40 0x7f2f8c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:46:07.311 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.315+0000 7f2fa57e7700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2f8c0a3c40 0x7f2f8c0a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0097b0 tx=0x7f2f9c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:46:07.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.318+0000 7f2f967fc700 1 -- 172.21.15.171:0/1953946821 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2f9001f020 con 0x7f2fa01068a0 2024-04-28T12:46:07.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:07 smithi171 ceph-mon[63127]: pgmap v197: 1 pgs: 1 unknown; 0 B data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:07.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:07 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi129 2024-04-28T12:46:07.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:07 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi129 to 5696M 2024-04-28T12:46:07.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:07 smithi171 ceph-mon[63127]: osdmap e53: 8 total, 8 up, 8 in 2024-04-28T12:46:07.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:07.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:07.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:07.314 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:07.476 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:07 smithi106 ceph-mon[28081]: pgmap v197: 1 pgs: 1 unknown; 0 B data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:07.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:07 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi129 2024-04-28T12:46:07.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:07 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi129 to 5696M 2024-04-28T12:46:07.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:07 smithi106 ceph-mon[28081]: osdmap e53: 8 total, 8 up, 8 in 2024-04-28T12:46:07.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:07.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:07.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:07.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:07 smithi129 ceph-mon[62418]: pgmap v197: 1 pgs: 1 unknown; 0 B data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:07 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi129 2024-04-28T12:46:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:07 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi129 to 5696M 2024-04-28T12:46:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:07 smithi129 ceph-mon[62418]: osdmap e53: 8 total, 8 up, 8 in 2024-04-28T12:46:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:07.525 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:07.529+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2fa0063860 con 0x7f2f8c0a3c40 2024-04-28T12:46:08.473 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:08 smithi171 ceph-mon[63127]: osdmap e54: 8 total, 8 up, 8 in 2024-04-28T12:46:08.473 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:08 smithi171 ceph-mon[63127]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:46:08.473 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:46:08.473 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:46:08.473 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:08.473 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:08 smithi171 ceph-mon[63127]: pgmap v200: 1 pgs: 1 unknown; 0 B data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:08.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:08 smithi129 ceph-mon[62418]: osdmap e54: 8 total, 8 up, 8 in 2024-04-28T12:46:08.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:08 smithi129 ceph-mon[62418]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:46:08.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:46:08.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:46:08.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:08.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:08 smithi129 ceph-mon[62418]: pgmap v200: 1 pgs: 1 unknown; 0 B data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:08.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:08 smithi106 ceph-mon[28081]: osdmap e54: 8 total, 8 up, 8 in 2024-04-28T12:46:08.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:08 smithi106 ceph-mon[28081]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:46:08.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:46:08.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:46:08.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:08.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:08 smithi106 ceph-mon[28081]: pgmap v200: 1 pgs: 1 unknown; 0 B data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:09.437 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:09 smithi171 ceph-mon[63127]: osdmap e55: 8 total, 8 up, 8 in 2024-04-28T12:46:09.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:09 smithi106 ceph-mon[28081]: osdmap e55: 8 total, 8 up, 8 in 2024-04-28T12:46:09.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:09 smithi129 ceph-mon[62418]: osdmap e55: 8 total, 8 up, 8 in 2024-04-28T12:46:10.666 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:10 smithi171 ceph-mon[63127]: pgmap v202: 1 pgs: 1 unknown; 0 B data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:10.666 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:10 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/1737084597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf9f2b4-6bf1-4740-8170-456e3b3f858a"}]: dispatch 2024-04-28T12:46:10.666 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:10 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf9f2b4-6bf1-4740-8170-456e3b3f858a"}]: dispatch 2024-04-28T12:46:10.666 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:10 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cf9f2b4-6bf1-4740-8170-456e3b3f858a"}]': finished 2024-04-28T12:46:10.667 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:10 smithi171 ceph-mon[63127]: osdmap e56: 9 total, 8 up, 9 in 2024-04-28T12:46:10.667 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:10.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:10 smithi129 ceph-mon[62418]: pgmap v202: 1 pgs: 1 unknown; 0 B data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:10.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:10 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/1737084597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf9f2b4-6bf1-4740-8170-456e3b3f858a"}]: dispatch 2024-04-28T12:46:10.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:10 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf9f2b4-6bf1-4740-8170-456e3b3f858a"}]: dispatch 2024-04-28T12:46:10.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:10 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cf9f2b4-6bf1-4740-8170-456e3b3f858a"}]': finished 2024-04-28T12:46:10.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:10 smithi129 ceph-mon[62418]: osdmap e56: 9 total, 8 up, 9 in 2024-04-28T12:46:10.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:10.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:10 smithi106 ceph-mon[28081]: pgmap v202: 1 pgs: 1 unknown; 0 B data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:10.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:10 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/1737084597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf9f2b4-6bf1-4740-8170-456e3b3f858a"}]: dispatch 2024-04-28T12:46:10.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:10 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf9f2b4-6bf1-4740-8170-456e3b3f858a"}]: dispatch 2024-04-28T12:46:10.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:10 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cf9f2b4-6bf1-4740-8170-456e3b3f858a"}]': finished 2024-04-28T12:46:10.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:10 smithi106 ceph-mon[28081]: osdmap e56: 9 total, 8 up, 9 in 2024-04-28T12:46:10.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:11.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:11 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/1139008327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:46:11.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:11 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/1139008327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:46:11.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:11 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/1139008327' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:46:12.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:12 smithi171 ceph-mon[63127]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:12.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:12 smithi129 ceph-mon[62418]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:12.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:12 smithi106 ceph-mon[28081]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:15.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:15 smithi106 ceph-mon[28081]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:15.352 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:15 smithi171 ceph-mon[63127]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:15.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:15 smithi129 ceph-mon[62418]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:17.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:17 smithi106 ceph-mon[28081]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:17.449 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:17 smithi171 ceph-mon[63127]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:17.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:17 smithi129 ceph-mon[62418]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:18.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-28T12:46:18.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:18.480 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:18 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-28T12:46:18.481 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:18 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:18.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:18 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-28T12:46:18.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:18 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:19.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:19 smithi171 ceph-mon[63127]: Deploying daemon osd.8 on smithi171 2024-04-28T12:46:19.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:19 smithi171 ceph-mon[63127]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:19.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:19 smithi106 ceph-mon[28081]: Deploying daemon osd.8 on smithi171 2024-04-28T12:46:19.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:19 smithi106 ceph-mon[28081]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:19.414 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:19 smithi129 ceph-mon[62418]: Deploying daemon osd.8 on smithi171 2024-04-28T12:46:19.415 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:19 smithi129 ceph-mon[62418]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:21.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:21 smithi171 ceph-mon[63127]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:21.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:21 smithi106 ceph-mon[28081]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:21.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:21 smithi129 ceph-mon[62418]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:23.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:23 smithi106 ceph-mon[28081]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:23.413 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:23 smithi171 ceph-mon[63127]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:23.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:23 smithi129 ceph-mon[62418]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:24.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:24.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:24.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:24.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:24 smithi171 ceph-mon[63127]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:25.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:25.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:25.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:25.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:24 smithi129 ceph-mon[62418]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:25.074 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:25.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:25.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:25.075 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:24 smithi106 ceph-mon[28081]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:27.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:27 smithi171 ceph-mon[63127]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:27.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:27.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:27.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:27.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:27.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:27 smithi106 ceph-mon[28081]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:27 smithi129 ceph-mon[62418]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:27.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:27.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:27.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:27.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:27.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:27.712 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 8 on host 'smithi171' 2024-04-28T12:46:27.712 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.716+0000 7f2f967fc700 1 -- 172.21.15.171:0/1953946821 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2fa0063860 con 0x7f2f8c0a3c40 2024-04-28T12:46:27.714 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2f8c0a3c40 msgr2=0x7f2f8c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:27.714 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2f8c0a3c40 0x7f2f8c0a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0097b0 tx=0x7f2f9c006d20 comp rx=0 tx=0).stop 2024-04-28T12:46:27.714 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01068a0 msgr2=0x7f2fa0199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:27.714 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01068a0 0x7f2fa0199bf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000b560 tx=0x7f2f9000b870 comp rx=0 tx=0).stop 2024-04-28T12:46:27.714 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 shutdown_connections 2024-04-28T12:46:27.714 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fa01091d0 0x7f2fa01a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:27.715 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fa0103f70 0x7f2fa01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:27.715 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2f8c0a3c40 0x7f2f8c0a6100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:27.715 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 --2- 172.21.15.171:0/1953946821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa01068a0 0x7f2fa0199bf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:27.715 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 >> 172.21.15.171:0/1953946821 conn(0x7f2fa00fd940 msgr2=0x7f2fa01048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:46:27.715 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 shutdown_connections 2024-04-28T12:46:27.715 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:46:27.718+0000 7f2fa7a4b700 1 -- 172.21.15.171:0/1953946821 wait complete. 2024-04-28T12:46:28.302 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:28 smithi171 ceph-mon[63127]: from='osd.8 [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-28T12:46:28.302 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:28 smithi171 ceph-mon[63127]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-28T12:46:28.303 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:28 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:28.303 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:28 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:28.303 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:28 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:28 smithi106 ceph-mon[28081]: from='osd.8 [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-28T12:46:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:28 smithi106 ceph-mon[28081]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-28T12:46:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:28 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:28 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:28 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:28.329 DEBUG:teuthology.orchestra.run.smithi171:osd.8> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.8.service 2024-04-28T12:46:28.331 INFO:tasks.cephadm:Deploying osd.9 on smithi106 with /dev/vg_nvme/lv_1... 2024-04-28T12:46:28.331 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-28T12:46:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:28 smithi129 ceph-mon[62418]: from='osd.8 [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-28T12:46:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:28 smithi129 ceph-mon[62418]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-28T12:46:28.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:28 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:28.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:28 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:28.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:28 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:28.779 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:46:29.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:29 smithi106 ceph-mon[28081]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:29.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:29 smithi106 ceph-mon[28081]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-28T12:46:29.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:29 smithi106 ceph-mon[28081]: from='osd.8 [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:46:29.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:29 smithi106 ceph-mon[28081]: osdmap e57: 9 total, 8 up, 9 in 2024-04-28T12:46:29.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:29 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:29.248 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:29 smithi106 ceph-mon[28081]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:46:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:29 smithi171 ceph-mon[63127]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:29 smithi171 ceph-mon[63127]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-28T12:46:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:29 smithi171 ceph-mon[63127]: from='osd.8 [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:46:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:29 smithi171 ceph-mon[63127]: osdmap e57: 9 total, 8 up, 9 in 2024-04-28T12:46:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:29 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:29.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:29 smithi171 ceph-mon[63127]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:46:29.496 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 12:46:29 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T12:46:29.078+0000 7ff82e045700 -1 osd.8 0 waiting for initial osdmap 2024-04-28T12:46:29.496 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 12:46:29 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T12:46:29.089+0000 7ff829e38700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:46:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:29 smithi129 ceph-mon[62418]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:29 smithi129 ceph-mon[62418]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-28T12:46:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:29 smithi129 ceph-mon[62418]: from='osd.8 [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:46:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:29 smithi129 ceph-mon[62418]: osdmap e57: 9 total, 8 up, 9 in 2024-04-28T12:46:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:29 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:29 smithi129 ceph-mon[62418]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:46:30.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:30 smithi171 ceph-mon[63127]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:46:30.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:30 smithi171 ceph-mon[63127]: osdmap e58: 9 total, 8 up, 9 in 2024-04-28T12:46:30.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:30 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:30.483 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:30 smithi106 ceph-mon[28081]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:46:30.483 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:30 smithi106 ceph-mon[28081]: osdmap e58: 9 total, 8 up, 9 in 2024-04-28T12:46:30.483 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:30 smithi129 ceph-mon[62418]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:46:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:30 smithi129 ceph-mon[62418]: osdmap e58: 9 total, 8 up, 9 in 2024-04-28T12:46:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:30 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:31.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:31 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:46:31.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:31 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:46:31.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:31 smithi106 ceph-mon[28081]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:31.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:31 smithi106 ceph-mon[28081]: osd.8 [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] boot 2024-04-28T12:46:31.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:31 smithi106 ceph-mon[28081]: osdmap e59: 9 total, 9 up, 9 in 2024-04-28T12:46:31.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:31 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:31.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:31 smithi106 ceph-mon[28081]: osdmap e60: 9 total, 9 up, 9 in 2024-04-28T12:46:31.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:31 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:46:31.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:31 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:46:31.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:31 smithi171 ceph-mon[63127]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:31.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:31 smithi171 ceph-mon[63127]: osd.8 [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] boot 2024-04-28T12:46:31.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:31 smithi171 ceph-mon[63127]: osdmap e59: 9 total, 9 up, 9 in 2024-04-28T12:46:31.453 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:31 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:31.454 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:31 smithi171 ceph-mon[63127]: osdmap e60: 9 total, 9 up, 9 in 2024-04-28T12:46:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:31 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:46:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:31 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:46:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:31 smithi129 ceph-mon[62418]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-28T12:46:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:31 smithi129 ceph-mon[62418]: osd.8 [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] boot 2024-04-28T12:46:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:31 smithi129 ceph-mon[62418]: osdmap e59: 9 total, 9 up, 9 in 2024-04-28T12:46:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:31 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-28T12:46:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:31 smithi129 ceph-mon[62418]: osdmap e60: 9 total, 9 up, 9 in 2024-04-28T12:46:31.718 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-28T12:46:31.718 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-28T12:46:31.718 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-28T12:46:31.718 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-28T12:46:31.718 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0330392 s, 317 MB/s 2024-04-28T12:46:31.718 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-28T12:46:32.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_1 2024-04-28T12:46:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi171 2024-04-28T12:46:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-28T12:46:32.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:32.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:32.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:32.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:32.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: osdmap e61: 9 total, 9 up, 9 in 2024-04-28T12:46:32.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:32 smithi106 ceph-mon[28081]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:32.961 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:46:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi171 2024-04-28T12:46:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-28T12:46:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:32.995 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:32.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:32.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:32.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: osdmap e61: 9 total, 9 up, 9 in 2024-04-28T12:46:32.996 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:32 smithi171 ceph-mon[63127]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi171 2024-04-28T12:46:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:46:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi171 to 5696M 2024-04-28T12:46:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:33.013 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:33.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:46:33.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:33.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: osdmap e61: 9 total, 9 up, 9 in 2024-04-28T12:46:33.014 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:32 smithi129 ceph-mon[62418]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:34.138 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:33 smithi106 ceph-mon[28081]: osdmap e62: 9 total, 9 up, 9 in 2024-04-28T12:46:34.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:33 smithi171 ceph-mon[63127]: osdmap e62: 9 total, 9 up, 9 in 2024-04-28T12:46:34.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:33 smithi129 ceph-mon[62418]: osdmap e62: 9 total, 9 up, 9 in 2024-04-28T12:46:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.130+0000 7f8966400700 1 -- 172.21.15.106:0/202812462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 msgr2=0x7f896010b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.130+0000 7f8966400700 1 --2- 172.21.15.106:0/202812462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 0x7f896010b7f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f895800b0d0 tx=0x7f895800b3e0 comp rx=0 tx=0).stop 2024-04-28T12:46:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.130+0000 7f8966400700 1 -- 172.21.15.106:0/202812462 shutdown_connections 2024-04-28T12:46:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.130+0000 7f8966400700 1 --2- 172.21.15.106:0/202812462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 0x7f896010b7f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.130+0000 7f8966400700 1 --2- 172.21.15.106:0/202812462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89601033a0 0x7f89601037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.130+0000 7f8966400700 1 --2- 172.21.15.106:0/202812462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89601045a0 0x7f8960104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.130+0000 7f8966400700 1 -- 172.21.15.106:0/202812462 >> 172.21.15.106:0/202812462 conn(0x7f89600fe960 msgr2=0x7f8960100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:46:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.130+0000 7f8966400700 1 -- 172.21.15.106:0/202812462 shutdown_connections 2024-04-28T12:46:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.130+0000 7f8966400700 1 -- 172.21.15.106:0/202812462 wait complete. 2024-04-28T12:46:35.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.131+0000 7f8966400700 1 Processor -- start 2024-04-28T12:46:35.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8966400700 1 -- start start 2024-04-28T12:46:35.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8966400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89601033a0 0x7f896019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:46:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8966400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89601045a0 0x7f896019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:46:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8966400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 0x7f89601a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:46:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8966400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f896010ee20 con 0x7f8960104f40 2024-04-28T12:46:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8966400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f896010eca0 con 0x7f89601045a0 2024-04-28T12:46:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8966400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f896010efa0 con 0x7f89601033a0 2024-04-28T12:46:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8965a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 0x7f89601a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:46:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8965000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89601033a0 0x7f896019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:46:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8965a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 0x7f89601a2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53156/0 (socket says 172.21.15.106:53156) 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f8965a00700 1 -- 172.21.15.106:0/2541476664 learned_addr learned my addr 172.21.15.106:0/2541476664 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.132+0000 7f895fe00700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89601045a0 0x7f896019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f8965000700 1 -- 172.21.15.106:0/2541476664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89601045a0 msgr2=0x7f896019bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f8965000700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89601045a0 0x7f896019bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f8965000700 1 -- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 msgr2=0x7f89601a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f8965000700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 0x7f89601a2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f8965000700 1 -- 172.21.15.106:0/2541476664 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89601a3280 con 0x7f89601033a0 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f8965a00700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 0x7f89601a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f8965000700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89601033a0 0x7f896019b620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f895400e9f0 tx=0x7f895400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f895d600700 1 -- 172.21.15.106:0/2541476664 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954004220 con 0x7f89601033a0 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f895d600700 1 -- 172.21.15.106:0/2541476664 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89540043c0 con 0x7f89601033a0 2024-04-28T12:46:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f895d600700 1 -- 172.21.15.106:0/2541476664 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954003be0 con 0x7f89601033a0 2024-04-28T12:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.133+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89601a3570 con 0x7f89601033a0 2024-04-28T12:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.134+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89601a3ab0 con 0x7f89601033a0 2024-04-28T12:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.135+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896006a110 con 0x7f89601033a0 2024-04-28T12:46:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.137+0000 7f895d600700 1 -- 172.21.15.106:0/2541476664 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f895400c7c0 con 0x7f89601033a0 2024-04-28T12:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.138+0000 7f895d600700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f89480a3c40 0x7f89480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.138+0000 7f895d600700 1 -- 172.21.15.106:0/2541476664 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f89540cfec0 con 0x7f89601033a0 2024-04-28T12:46:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.138+0000 7f895fe00700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f89480a3c40 0x7f89480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:46:35.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.139+0000 7f895fe00700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f89480a3c40 0x7f89480a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f894c009c60 tx=0x7f894c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:46:35.136 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:34 smithi106 ceph-mon[28081]: osdmap e63: 9 total, 9 up, 9 in 2024-04-28T12:46:35.136 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:34 smithi106 ceph-mon[28081]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.141+0000 7f895d600700 1 -- 172.21.15.106:0/2541476664 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f895409d1b0 con 0x7f89601033a0 2024-04-28T12:46:35.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:34 smithi171 ceph-mon[63127]: osdmap e63: 9 total, 9 up, 9 in 2024-04-28T12:46:35.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:34 smithi171 ceph-mon[63127]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:35.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:34 smithi129 ceph-mon[62418]: osdmap e63: 9 total, 9 up, 9 in 2024-04-28T12:46:35.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:34 smithi129 ceph-mon[62418]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:35.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:46:35.379+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f89601a3d70 con 0x7f89480a3c40 2024-04-28T12:46:36.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:46:36.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:46:36.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:36.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:46:36.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:46:36.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:36.314 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:46:36.314 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:46:36.314 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:37.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:36 smithi171 ceph-mon[63127]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:46:37.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:36 smithi171 ceph-mon[63127]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-28T12:46:37.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:36 smithi129 ceph-mon[62418]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:46:37.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:36 smithi129 ceph-mon[62418]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-28T12:46:37.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:36 smithi106 ceph-mon[28081]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:46:37.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:36 smithi106 ceph-mon[28081]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-28T12:46:39.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:39 smithi106 ceph-mon[28081]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-28T12:46:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:39 smithi171 ceph-mon[63127]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-28T12:46:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:39 smithi129 ceph-mon[62418]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-28T12:46:40.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:40 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3671111996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd4886f6-691b-4475-9180-11d5262b59a5"}]: dispatch 2024-04-28T12:46:40.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:40 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3671111996' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd4886f6-691b-4475-9180-11d5262b59a5"}]': finished 2024-04-28T12:46:40.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:40 smithi106 ceph-mon[28081]: osdmap e64: 10 total, 9 up, 10 in 2024-04-28T12:46:40.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:40 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:46:40.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:40 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3671111996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd4886f6-691b-4475-9180-11d5262b59a5"}]: dispatch 2024-04-28T12:46:40.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:40 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3671111996' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd4886f6-691b-4475-9180-11d5262b59a5"}]': finished 2024-04-28T12:46:40.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:40 smithi171 ceph-mon[63127]: osdmap e64: 10 total, 9 up, 10 in 2024-04-28T12:46:40.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:40 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:46:40.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:40 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3671111996' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd4886f6-691b-4475-9180-11d5262b59a5"}]: dispatch 2024-04-28T12:46:40.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:40 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3671111996' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd4886f6-691b-4475-9180-11d5262b59a5"}]': finished 2024-04-28T12:46:40.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:40 smithi129 ceph-mon[62418]: osdmap e64: 10 total, 9 up, 10 in 2024-04-28T12:46:40.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:40 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:46:41.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:41 smithi106 ceph-mon[28081]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-28T12:46:41.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:41 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3599587004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:46:41.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:41 smithi171 ceph-mon[63127]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-28T12:46:41.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:41 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3599587004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:46:41.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:41 smithi129 ceph-mon[62418]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-28T12:46:41.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:41 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3599587004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:46:43.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:43 smithi106 ceph-mon[28081]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-28T12:46:43.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:43 smithi171 ceph-mon[63127]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-28T12:46:43.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:43 smithi129 ceph-mon[62418]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-04-28T12:46:45.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:45 smithi106 ceph-mon[28081]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-28T12:46:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:45 smithi171 ceph-mon[63127]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-28T12:46:45.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:45 smithi129 ceph-mon[62418]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-28T12:46:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:47 smithi106 ceph-mon[28081]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:47.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:47 smithi171 ceph-mon[63127]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:47 smithi129 ceph-mon[62418]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:49 smithi106 ceph-mon[28081]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:49.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-28T12:46:49.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:49.329 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:49 smithi129 ceph-mon[62418]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:49.329 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-28T12:46:49.329 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:49 smithi171 ceph-mon[63127]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-28T12:46:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:46:50.366 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:50 smithi106 ceph-mon[28081]: Deploying daemon osd.9 on smithi106 2024-04-28T12:46:50.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:50 smithi171 ceph-mon[63127]: Deploying daemon osd.9 on smithi106 2024-04-28T12:46:50.505 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:50 smithi129 ceph-mon[62418]: Deploying daemon osd.9 on smithi106 2024-04-28T12:46:51.178 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:51 smithi106 ceph-mon[28081]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:51.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:51 smithi171 ceph-mon[63127]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:51 smithi129 ceph-mon[62418]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:53.281 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:53 smithi106 ceph-mon[28081]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:53.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:53 smithi171 ceph-mon[63127]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:53.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:53 smithi129 ceph-mon[62418]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:55.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:55 smithi106 ceph-mon[28081]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:55 smithi171 ceph-mon[63127]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:55 smithi129 ceph-mon[62418]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:57.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:57 smithi106 ceph-mon[28081]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:57.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:57.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:57.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:57 smithi171 ceph-mon[63127]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:57 smithi129 ceph-mon[62418]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:46:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:46:59.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:46:59 smithi171 ceph-mon[63127]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:59.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:46:59 smithi129 ceph-mon[62418]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:46:59.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:46:59 smithi106 ceph-mon[28081]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:47:01.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:01 smithi106 ceph-mon[28081]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:47:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:01 smithi106 ceph-mon[28081]: from='osd.9 [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-28T12:47:01.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:01 smithi106 ceph-mon[28081]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-28T12:47:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:01 smithi171 ceph-mon[63127]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:47:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:01 smithi171 ceph-mon[63127]: from='osd.9 [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-28T12:47:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:01 smithi171 ceph-mon[63127]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-28T12:47:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:01 smithi129 ceph-mon[62418]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:47:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:01 smithi129 ceph-mon[62418]: from='osd.9 [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-28T12:47:01.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:01 smithi129 ceph-mon[62418]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-28T12:47:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.539+0000 7f895d600700 1 -- 172.21.15.106:0/2541476664 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f89601a3d70 con 0x7f89480a3c40 2024-04-28T12:47:01.535 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 9 on host 'smithi106' 2024-04-28T12:47:01.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.543+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f89480a3c40 msgr2=0x7f89480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:01.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.544+0000 7f8966400700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f89480a3c40 0x7f89480a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f894c009c60 tx=0x7f894c009260 comp rx=0 tx=0).stop 2024-04-28T12:47:01.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.544+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89601033a0 msgr2=0x7f896019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:01.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.544+0000 7f8966400700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89601033a0 0x7f896019b620 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f895400e9f0 tx=0x7f895400ed00 comp rx=0 tx=0).stop 2024-04-28T12:47:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.544+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 shutdown_connections 2024-04-28T12:47:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.544+0000 7f8966400700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8960104f40 0x7f89601a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.544+0000 7f8966400700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f89480a3c40 0x7f89480a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.544+0000 7f8966400700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89601033a0 0x7f896019b620 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.544+0000 7f8966400700 1 --2- 172.21.15.106:0/2541476664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89601045a0 0x7f896019bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.544+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 >> 172.21.15.106:0/2541476664 conn(0x7f89600fe960 msgr2=0x7f8960075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:47:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.545+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 shutdown_connections 2024-04-28T12:47:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:01.545+0000 7f8966400700 1 -- 172.21.15.106:0/2541476664 wait complete. 2024-04-28T12:47:02.301 DEBUG:teuthology.orchestra.run.smithi106:osd.9> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.9.service 2024-04-28T12:47:02.346 INFO:tasks.cephadm:Deploying osd.10 on smithi129 with /dev/vg_nvme/lv_1... 2024-04-28T12:47:02.346 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-28T12:47:02.561 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:02 smithi106 ceph-mon[28081]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-28T12:47:02.561 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:02 smithi106 ceph-mon[28081]: from='osd.9 [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:47:02.561 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:02 smithi106 ceph-mon[28081]: osdmap e65: 10 total, 9 up, 10 in 2024-04-28T12:47:02.561 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:02.561 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:02 smithi106 ceph-mon[28081]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:47:02.561 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:02.561 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:02.562 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:02 smithi106 ceph-mon[28081]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:47:02.590 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.b/config 2024-04-28T12:47:02.712 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:02 smithi129 ceph-mon[62418]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-28T12:47:02.712 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:02 smithi129 ceph-mon[62418]: from='osd.9 [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:47:02.712 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:02 smithi129 ceph-mon[62418]: osdmap e65: 10 total, 9 up, 10 in 2024-04-28T12:47:02.713 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:02.713 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:02 smithi129 ceph-mon[62418]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:47:02.713 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:02.713 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:02.713 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:02 smithi129 ceph-mon[62418]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:47:02.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:02 smithi171 ceph-mon[63127]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-28T12:47:02.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:02 smithi171 ceph-mon[63127]: from='osd.9 [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:47:02.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:02 smithi171 ceph-mon[63127]: osdmap e65: 10 total, 9 up, 10 in 2024-04-28T12:47:02.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:02.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:02 smithi171 ceph-mon[63127]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-28T12:47:02.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:02.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:02.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:02 smithi171 ceph-mon[63127]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-28T12:47:03.737 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:03 smithi129 ceph-mon[62418]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:47:03.737 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:03 smithi129 ceph-mon[62418]: osdmap e66: 10 total, 9 up, 10 in 2024-04-28T12:47:03.737 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:03 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:03.737 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:03 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:03.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:03 smithi171 ceph-mon[63127]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:47:03.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:03 smithi171 ceph-mon[63127]: osdmap e66: 10 total, 9 up, 10 in 2024-04-28T12:47:03.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:03 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:03.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:03 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:03.758 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:03 smithi106 ceph-mon[28081]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-28T12:47:03.758 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:03 smithi106 ceph-mon[28081]: osdmap e66: 10 total, 9 up, 10 in 2024-04-28T12:47:03.758 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:03 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:03.758 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:03 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:04.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:04 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:47:04.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:04 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:47:04.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:04 smithi171 ceph-mon[63127]: osd.9 [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] boot 2024-04-28T12:47:04.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:04 smithi171 ceph-mon[63127]: osdmap e67: 10 total, 10 up, 10 in 2024-04-28T12:47:04.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:04.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:04 smithi171 ceph-mon[63127]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:04.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:04 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:47:04.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:04 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:47:04.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:04 smithi129 ceph-mon[62418]: osd.9 [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] boot 2024-04-28T12:47:04.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:04 smithi129 ceph-mon[62418]: osdmap e67: 10 total, 10 up, 10 in 2024-04-28T12:47:04.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:04 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:04.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:04 smithi129 ceph-mon[62418]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-28T12:47:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-28T12:47:04.764 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10+0 records in 2024-04-28T12:47:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:10+0 records out 2024-04-28T12:47:04.764 INFO:teuthology.orchestra.run.smithi129.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112862 s, 929 MB/s 2024-04-28T12:47:04.764 INFO:teuthology.orchestra.run.smithi129.stderr:--> Zapping successful for: 2024-04-28T12:47:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:04 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:47:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:04 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:47:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:04 smithi106 ceph-mon[28081]: osd.9 [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] boot 2024-04-28T12:47:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:04 smithi106 ceph-mon[28081]: osdmap e67: 10 total, 10 up, 10 in 2024-04-28T12:47:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-28T12:47:04.825 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:04 smithi106 ceph-mon[28081]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:05.458 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:05 smithi129 ceph-mon[62418]: osdmap e68: 10 total, 10 up, 10 in 2024-04-28T12:47:05.486 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_1 2024-04-28T12:47:05.732 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.b/config 2024-04-28T12:47:05.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:05 smithi171 ceph-mon[63127]: osdmap e68: 10 total, 10 up, 10 in 2024-04-28T12:47:05.824 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:05 smithi106 ceph-mon[28081]: osdmap e68: 10 total, 10 up, 10 in 2024-04-28T12:47:06.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:06 smithi129 ceph-mon[62418]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:06.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:06 smithi171 ceph-mon[63127]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:06.817 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:06 smithi106 ceph-mon[28081]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:07.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.447+0000 7f238bdbd700 1 -- 172.21.15.129:0/3024036568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 msgr2=0x7f2384104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.447+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3024036568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 0x7f2384104ae0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f23800099c0 tx=0x7f2380009cd0 comp rx=0 tx=0).stop 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.447+0000 7f238bdbd700 1 -- 172.21.15.129:0/3024036568 shutdown_connections 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.447+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3024036568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2384105020 0x7f238410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.447+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3024036568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2384103480 0x7f23841038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.447+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3024036568 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 0x7f2384104ae0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.447+0000 7f238bdbd700 1 -- 172.21.15.129:0/3024036568 >> 172.21.15.129:0/3024036568 conn(0x7f23840fea20 msgr2=0x7f2384100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 -- 172.21.15.129:0/3024036568 shutdown_connections 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 -- 172.21.15.129:0/3024036568 wait complete. 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 Processor -- start 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 -- start start 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2384103480 0x7f23840771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:07.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 0x7f2384075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2384105020 0x7f2384075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f238410ef90 con 0x7f2384105020 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f238410ee10 con 0x7f2384104680 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.448+0000 7f238bdbd700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f238410f110 con 0x7f2384103480 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389358700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 0x7f2384075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389358700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 0x7f2384075810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46018/0 (socket says 172.21.15.129:46018) 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389358700 1 -- 172.21.15.129:0/3730499693 learned_addr learned my addr 172.21.15.129:0/3730499693 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389b59700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2384103480 0x7f23840771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389358700 1 -- 172.21.15.129:0/3730499693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2384103480 msgr2=0x7f23840771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389358700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2384103480 0x7f23840771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389358700 1 -- 172.21.15.129:0/3730499693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2384105020 msgr2=0x7f2384075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:07.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389358700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2384105020 0x7f2384075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:07.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389358700 1 -- 172.21.15.129:0/3730499693 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2384076290 con 0x7f2384104680 2024-04-28T12:47:07.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f2389358700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 0x7f2384075810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2380005910 tx=0x7f238000fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:47:07.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f237affd700 1 -- 172.21.15.129:0/3730499693 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380027070 con 0x7f2384104680 2024-04-28T12:47:07.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f237affd700 1 -- 172.21.15.129:0/3730499693 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2380022920 con 0x7f2384104680 2024-04-28T12:47:07.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2384076520 con 0x7f2384104680 2024-04-28T12:47:07.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f237affd700 1 -- 172.21.15.129:0/3730499693 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380009e60 con 0x7f2384104680 2024-04-28T12:47:07.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.449+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23841a8090 con 0x7f2384104680 2024-04-28T12:47:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.450+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238406a110 con 0x7f2384104680 2024-04-28T12:47:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.451+0000 7f237affd700 1 -- 172.21.15.129:0/3730499693 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f238002a070 con 0x7f2384104680 2024-04-28T12:47:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.452+0000 7f237affd700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f23700a3c40 0x7f23700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.452+0000 7f237affd700 1 -- 172.21.15.129:0/3730499693 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f23800d0c70 con 0x7f2384104680 2024-04-28T12:47:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.452+0000 7f2389b59700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f23700a3c40 0x7f23700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:07.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.453+0000 7f2389b59700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f23700a3c40 0x7f23700a6100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f23841044e0 tx=0x7f2374009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:47:07.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.455+0000 7f237affd700 1 -- 172.21.15.129:0/3730499693 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2380013070 con 0x7f2384104680 2024-04-28T12:47:07.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:07.663+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2384063860 con 0x7f23700a3c40 2024-04-28T12:47:08.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi106 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:08.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:08.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:47:08.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:47:08.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi106 2024-04-28T12:47:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:47:08.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi106 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:47:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:47:08.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:09.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:08 smithi171 ceph-mon[63127]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:47:09.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:08 smithi171 ceph-mon[63127]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:08 smithi129 ceph-mon[62418]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:47:09.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:08 smithi129 ceph-mon[62418]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:09.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:08 smithi106 ceph-mon[28081]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:47:09.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:08 smithi106 ceph-mon[28081]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:11.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:11 smithi129 ceph-mon[62418]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:11.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:11 smithi129 ceph-mon[62418]: from='client.? 172.21.15.129:0/4237015674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d8f70b-cc2e-43d1-b34f-d487b001e922"}]: dispatch 2024-04-28T12:47:11.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:11 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d8f70b-cc2e-43d1-b34f-d487b001e922"}]: dispatch 2024-04-28T12:47:11.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:11 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2d8f70b-cc2e-43d1-b34f-d487b001e922"}]': finished 2024-04-28T12:47:11.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:11 smithi129 ceph-mon[62418]: osdmap e69: 11 total, 10 up, 11 in 2024-04-28T12:47:11.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:11 smithi106 ceph-mon[28081]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:11 smithi106 ceph-mon[28081]: from='client.? 172.21.15.129:0/4237015674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d8f70b-cc2e-43d1-b34f-d487b001e922"}]: dispatch 2024-04-28T12:47:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:11 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d8f70b-cc2e-43d1-b34f-d487b001e922"}]: dispatch 2024-04-28T12:47:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:11 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2d8f70b-cc2e-43d1-b34f-d487b001e922"}]': finished 2024-04-28T12:47:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:11 smithi106 ceph-mon[28081]: osdmap e69: 11 total, 10 up, 11 in 2024-04-28T12:47:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:11 smithi171 ceph-mon[63127]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:11 smithi171 ceph-mon[63127]: from='client.? 172.21.15.129:0/4237015674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d8f70b-cc2e-43d1-b34f-d487b001e922"}]: dispatch 2024-04-28T12:47:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:11 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2d8f70b-cc2e-43d1-b34f-d487b001e922"}]: dispatch 2024-04-28T12:47:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:11 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2d8f70b-cc2e-43d1-b34f-d487b001e922"}]': finished 2024-04-28T12:47:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:11 smithi171 ceph-mon[63127]: osdmap e69: 11 total, 10 up, 11 in 2024-04-28T12:47:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:12.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:12 smithi171 ceph-mon[63127]: from='client.? 172.21.15.129:0/368715761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:47:12.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:12 smithi129 ceph-mon[62418]: from='client.? 172.21.15.129:0/368715761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:47:12.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:12 smithi106 ceph-mon[28081]: from='client.? 172.21.15.129:0/368715761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:47:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:13 smithi171 ceph-mon[63127]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:13 smithi129 ceph-mon[62418]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:13.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:13 smithi106 ceph-mon[28081]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:15.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:15 smithi129 ceph-mon[62418]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:15 smithi171 ceph-mon[63127]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:15.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:15 smithi106 ceph-mon[28081]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:17.476 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:17 smithi129 ceph-mon[62418]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:17.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:17 smithi171 ceph-mon[63127]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:17.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:17 smithi106 ceph-mon[28081]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:19.201 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:19 smithi129 ceph-mon[62418]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:19.201 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:19 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-28T12:47:19.201 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:19 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:19 smithi171 ceph-mon[63127]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:19 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-28T12:47:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:19 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:19.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:19 smithi106 ceph-mon[28081]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:19.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:19 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-28T12:47:19.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:19 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:20.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:20 smithi129 ceph-mon[62418]: Deploying daemon osd.10 on smithi129 2024-04-28T12:47:20.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:20 smithi171 ceph-mon[63127]: Deploying daemon osd.10 on smithi129 2024-04-28T12:47:20.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:20 smithi106 ceph-mon[28081]: Deploying daemon osd.10 on smithi129 2024-04-28T12:47:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:21 smithi171 ceph-mon[63127]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:21.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:21 smithi129 ceph-mon[62418]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:21.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:21 smithi106 ceph-mon[28081]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:23.398 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:23 smithi129 ceph-mon[62418]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:23 smithi171 ceph-mon[63127]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:23.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:23 smithi106 ceph-mon[28081]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:25.378 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:25 smithi129 ceph-mon[62418]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:25.378 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:25 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:25.378 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:25 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:25.378 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:25 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:25 smithi171 ceph-mon[63127]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:25 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:25 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:25 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:25.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:25 smithi106 ceph-mon[28081]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:25.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:25 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:25.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:25 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:25.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:25 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:27.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:27 smithi129 ceph-mon[62418]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:27 smithi171 ceph-mon[63127]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:27.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:27 smithi106 ceph-mon[28081]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:28.453 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 10 on host 'smithi129' 2024-04-28T12:47:28.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.456+0000 7f237affd700 1 -- 172.21.15.129:0/3730499693 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2384063860 con 0x7f23700a3c40 2024-04-28T12:47:28.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f23700a3c40 msgr2=0x7f23700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:28.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f23700a3c40 0x7f23700a6100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f23841044e0 tx=0x7f2374009260 comp rx=0 tx=0).stop 2024-04-28T12:47:28.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 msgr2=0x7f2384075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:28.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 0x7f2384075810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2380005910 tx=0x7f238000fab0 comp rx=0 tx=0).stop 2024-04-28T12:47:28.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 shutdown_connections 2024-04-28T12:47:28.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2384105020 0x7f2384075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:28.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f23700a3c40 0x7f23700a6100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:28.455 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2384103480 0x7f23840771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:28.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 --2- 172.21.15.129:0/3730499693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2384104680 0x7f2384075810 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:28.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 >> 172.21.15.129:0/3730499693 conn(0x7f23840fea20 msgr2=0x7f2384107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:47:28.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 shutdown_connections 2024-04-28T12:47:28.456 INFO:teuthology.orchestra.run.smithi129.stderr:2024-04-28T12:47:28.459+0000 7f238bdbd700 1 -- 172.21.15.129:0/3730499693 wait complete. 2024-04-28T12:47:29.173 DEBUG:teuthology.orchestra.run.smithi129:osd.10> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.10.service 2024-04-28T12:47:29.175 INFO:tasks.cephadm:Deploying osd.11 on smithi171 with /dev/vg_nvme/lv_1... 2024-04-28T12:47:29.176 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-28T12:47:29.399 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:29 smithi129 ceph-mon[62418]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:29.399 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:29 smithi129 ceph-mon[62418]: from='osd.10 [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-28T12:47:29.399 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:29 smithi129 ceph-mon[62418]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-28T12:47:29.399 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:29 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.399 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:29 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.399 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:29 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.399 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:29 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.425 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:47:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:29 smithi171 ceph-mon[63127]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:29 smithi171 ceph-mon[63127]: from='osd.10 [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-28T12:47:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:29 smithi171 ceph-mon[63127]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-28T12:47:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:29 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:29 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:29 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:29 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:29 smithi106 ceph-mon[28081]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:29.719 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:29 smithi106 ceph-mon[28081]: from='osd.10 [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-28T12:47:29.719 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:29 smithi106 ceph-mon[28081]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-28T12:47:29.720 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:29 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.720 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:29 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.720 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:29 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:29.720 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:29 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:30.482 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:30 smithi129 ceph-mon[62418]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-28T12:47:30.482 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:30 smithi129 ceph-mon[62418]: from='osd.10 [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:47:30.482 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:30 smithi129 ceph-mon[62418]: osdmap e70: 11 total, 10 up, 11 in 2024-04-28T12:47:30.482 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:30 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:30.482 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:30 smithi129 ceph-mon[62418]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:47:30.482 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 12:47:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T12:47:30.148+0000 7f6fd0cce700 -1 osd.10 0 waiting for initial osdmap 2024-04-28T12:47:30.483 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 12:47:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T12:47:30.163+0000 7f6fccac1700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:47:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:30 smithi171 ceph-mon[63127]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-28T12:47:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:30 smithi171 ceph-mon[63127]: from='osd.10 [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:47:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:30 smithi171 ceph-mon[63127]: osdmap e70: 11 total, 10 up, 11 in 2024-04-28T12:47:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:30 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:30 smithi171 ceph-mon[63127]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:47:30.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:30 smithi106 ceph-mon[28081]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-28T12:47:30.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:30 smithi106 ceph-mon[28081]: from='osd.10 [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:47:30.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:30 smithi106 ceph-mon[28081]: osdmap e70: 11 total, 10 up, 11 in 2024-04-28T12:47:30.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:30.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:30 smithi106 ceph-mon[28081]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-04-28T12:47:31.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:31 smithi129 ceph-mon[62418]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:31.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:31 smithi129 ceph-mon[62418]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:47:31.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:31 smithi129 ceph-mon[62418]: osdmap e71: 11 total, 10 up, 11 in 2024-04-28T12:47:31.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:31 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:31.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:31 smithi129 ceph-mon[62418]: osd.10 [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] boot 2024-04-28T12:47:31.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:31 smithi129 ceph-mon[62418]: osdmap e72: 11 total, 11 up, 11 in 2024-04-28T12:47:31.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:31 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:31.427 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:31 smithi171 ceph-mon[63127]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:31.427 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:31 smithi171 ceph-mon[63127]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:47:31.427 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:31 smithi171 ceph-mon[63127]: osdmap e71: 11 total, 10 up, 11 in 2024-04-28T12:47:31.427 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:31 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:31.427 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:31 smithi171 ceph-mon[63127]: osd.10 [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] boot 2024-04-28T12:47:31.427 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:31 smithi171 ceph-mon[63127]: osdmap e72: 11 total, 11 up, 11 in 2024-04-28T12:47:31.427 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:31 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:31.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:31 smithi106 ceph-mon[28081]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-28T12:47:31.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:31 smithi106 ceph-mon[28081]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-04-28T12:47:31.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:31 smithi106 ceph-mon[28081]: osdmap e71: 11 total, 10 up, 11 in 2024-04-28T12:47:31.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:31 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:31.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:31 smithi106 ceph-mon[28081]: osd.10 [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] boot 2024-04-28T12:47:31.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:31 smithi106 ceph-mon[28081]: osdmap e72: 11 total, 11 up, 11 in 2024-04-28T12:47:31.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:31 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-28T12:47:31.654 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-28T12:47:31.655 INFO:teuthology.orchestra.run.smithi171.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-28T12:47:31.655 INFO:teuthology.orchestra.run.smithi171.stderr: stderr: 10+0 records in 2024-04-28T12:47:31.655 INFO:teuthology.orchestra.run.smithi171.stderr:10+0 records out 2024-04-28T12:47:31.655 INFO:teuthology.orchestra.run.smithi171.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0152007 s, 690 MB/s 2024-04-28T12:47:31.655 INFO:teuthology.orchestra.run.smithi171.stderr:--> Zapping successful for: 2024-04-28T12:47:32.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:32 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:47:32.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:32 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:47:32.354 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:32 smithi171 ceph-mon[63127]: osdmap e73: 11 total, 11 up, 11 in 2024-04-28T12:47:32.382 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2024-04-28T12:47:32.382 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:32 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:47:32.383 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:32 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:47:32.383 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:32 smithi129 ceph-mon[62418]: osdmap e73: 11 total, 11 up, 11 in 2024-04-28T12:47:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:32 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:47:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:32 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:47:32.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:32 smithi106 ceph-mon[28081]: osdmap e73: 11 total, 11 up, 11 in 2024-04-28T12:47:32.625 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:47:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:33.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:33 smithi171 ceph-mon[63127]: osdmap e74: 11 total, 11 up, 11 in 2024-04-28T12:47:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:33 smithi129 ceph-mon[62418]: osdmap e74: 11 total, 11 up, 11 in 2024-04-28T12:47:33.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:33.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:47:33.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:33.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:33 smithi106 ceph-mon[28081]: osdmap e74: 11 total, 11 up, 11 in 2024-04-28T12:47:34.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.400+0000 7fcf757ba700 1 -- 172.21.15.171:0/2113666575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70106830 msgr2=0x7fcf70108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:34.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.400+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2113666575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70106830 0x7fcf70108c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009990 tx=0x7fcf58009ca0 comp rx=0 tx=0).stop 2024-04-28T12:47:34.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.400+0000 7fcf757ba700 1 -- 172.21.15.171:0/2113666575 shutdown_connections 2024-04-28T12:47:34.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.400+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2113666575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf70109160 0x7fcf7010b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:34.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.400+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2113666575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf70103f00 0x7fcf701062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:34.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.400+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2113666575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70106830 0x7fcf70108c20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:34.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.400+0000 7fcf757ba700 1 -- 172.21.15.171:0/2113666575 >> 172.21.15.171:0/2113666575 conn(0x7fcf700fd910 msgr2=0x7fcf700ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:47:34.396 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.400+0000 7fcf757ba700 1 -- 172.21.15.171:0/2113666575 shutdown_connections 2024-04-28T12:47:34.397 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.400+0000 7fcf757ba700 1 -- 172.21.15.171:0/2113666575 wait complete. 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf757ba700 1 Processor -- start 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf757ba700 1 -- start start 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf757ba700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf70103f00 0x7fcf7019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf757ba700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf70106830 0x7fcf7019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf757ba700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70109160 0x7fcf701a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf757ba700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010f040 con 0x7fcf70106830 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf757ba700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010eec0 con 0x7fcf70103f00 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf757ba700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7010f1c0 con 0x7fcf70109160 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf6f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70109160 0x7fcf701a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf6f7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70109160 0x7fcf701a2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:43634/0 (socket says 172.21.15.171:43634) 2024-04-28T12:47:34.399 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf6f7fe700 1 -- 172.21.15.171:0/2281103522 learned_addr learned my addr 172.21.15.171:0/2281103522 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf6effd700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf70103f00 0x7fcf7019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf6f7fe700 1 -- 172.21.15.171:0/2281103522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf70103f00 msgr2=0x7fcf7019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf6f7fe700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf70103f00 0x7fcf7019b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf6f7fe700 1 -- 172.21.15.171:0/2281103522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf70106830 msgr2=0x7fcf7019bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf6f7fe700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf70106830 0x7fcf7019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.401+0000 7fcf6f7fe700 1 -- 172.21.15.171:0/2281103522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf701a35c0 con 0x7fcf70109160 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.402+0000 7fcf6f7fe700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70109160 0x7fcf701a2f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800ba10 tx=0x7fcf6800bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.402+0000 7fcf6cff9700 1 -- 172.21.15.171:0/2281103522 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68019070 con 0x7fcf70109160 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.402+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf701a38b0 con 0x7fcf70109160 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.402+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf701a3ec0 con 0x7fcf70109160 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.402+0000 7fcf6cff9700 1 -- 172.21.15.171:0/2281103522 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf68014440 con 0x7fcf70109160 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.402+0000 7fcf6cff9700 1 -- 172.21.15.171:0/2281103522 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800ce60 con 0x7fcf70109160 2024-04-28T12:47:34.400 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.403+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7010ace0 con 0x7fcf70109160 2024-04-28T12:47:34.401 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.404+0000 7fcf6cff9700 1 -- 172.21.15.171:0/2281103522 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fcf6801c070 con 0x7fcf70109160 2024-04-28T12:47:34.401 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.404+0000 7fcf6cff9700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:34.401 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.405+0000 7fcf6cff9700 1 -- 172.21.15.171:0/2281103522 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7fcf680cf1c0 con 0x7fcf70109160 2024-04-28T12:47:34.401 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.405+0000 7fcf6effd700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:34.402 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.406+0000 7fcf6effd700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf60006fd0 tx=0x7fcf60008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:47:34.405 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.409+0000 7fcf6cff9700 1 -- 172.21.15.171:0/2281103522 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcf6809c1e0 con 0x7fcf70109160 2024-04-28T12:47:34.405 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:34 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi129 2024-04-28T12:47:34.405 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:34 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi129 to 4272M 2024-04-28T12:47:34.405 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:34 smithi171 ceph-mon[63127]: osdmap e75: 11 total, 11 up, 11 in 2024-04-28T12:47:34.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:34 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi129 2024-04-28T12:47:34.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:34 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi129 to 4272M 2024-04-28T12:47:34.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:34 smithi129 ceph-mon[62418]: osdmap e75: 11 total, 11 up, 11 in 2024-04-28T12:47:34.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:34 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi129 2024-04-28T12:47:34.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:34 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi129 to 4272M 2024-04-28T12:47:34.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:34 smithi106 ceph-mon[28081]: osdmap e75: 11 total, 11 up, 11 in 2024-04-28T12:47:34.610 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:34.614+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcf70063860 con 0x7fcf5c0a3cc0 2024-04-28T12:47:35.414 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:35 smithi171 ceph-mon[63127]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:35.414 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:47:35.414 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:47:35.415 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:35.415 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:35 smithi171 ceph-mon[63127]: osdmap e76: 11 total, 11 up, 11 in 2024-04-28T12:47:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:35 smithi129 ceph-mon[62418]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:47:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:47:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:35 smithi129 ceph-mon[62418]: osdmap e76: 11 total, 11 up, 11 in 2024-04-28T12:47:35.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:35 smithi106 ceph-mon[28081]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:35.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-28T12:47:35.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-28T12:47:35.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:35.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:35 smithi106 ceph-mon[28081]: osdmap e76: 11 total, 11 up, 11 in 2024-04-28T12:47:36.488 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:36 smithi171 ceph-mon[63127]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:47:36.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:36 smithi129 ceph-mon[62418]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:47:36.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:36 smithi106 ceph-mon[28081]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:47:37.476 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:37 smithi106 ceph-mon[28081]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-04-28T12:47:37.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:37 smithi171 ceph-mon[63127]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-04-28T12:47:37.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:37 smithi129 ceph-mon[62418]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-04-28T12:47:38.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:38 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/420522508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc45af10-78d3-4da4-b980-e7fde882529a"}]: dispatch 2024-04-28T12:47:38.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:38 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc45af10-78d3-4da4-b980-e7fde882529a"}]: dispatch 2024-04-28T12:47:38.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:38 smithi171 ceph-mon[63127]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc45af10-78d3-4da4-b980-e7fde882529a"}]': finished 2024-04-28T12:47:38.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:38 smithi171 ceph-mon[63127]: osdmap e77: 12 total, 11 up, 12 in 2024-04-28T12:47:38.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:38 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:38.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:38 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/420522508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc45af10-78d3-4da4-b980-e7fde882529a"}]: dispatch 2024-04-28T12:47:38.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:38 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc45af10-78d3-4da4-b980-e7fde882529a"}]: dispatch 2024-04-28T12:47:38.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:38 smithi129 ceph-mon[62418]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc45af10-78d3-4da4-b980-e7fde882529a"}]': finished 2024-04-28T12:47:38.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:38 smithi129 ceph-mon[62418]: osdmap e77: 12 total, 11 up, 12 in 2024-04-28T12:47:38.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:38 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:38.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:38 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/420522508' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc45af10-78d3-4da4-b980-e7fde882529a"}]: dispatch 2024-04-28T12:47:38.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:38 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc45af10-78d3-4da4-b980-e7fde882529a"}]: dispatch 2024-04-28T12:47:38.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:38 smithi106 ceph-mon[28081]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc45af10-78d3-4da4-b980-e7fde882529a"}]': finished 2024-04-28T12:47:38.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:38 smithi106 ceph-mon[28081]: osdmap e77: 12 total, 11 up, 12 in 2024-04-28T12:47:38.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:38 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:39 smithi171 ceph-mon[63127]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-28T12:47:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:39 smithi171 ceph-mon[63127]: from='client.? 172.21.15.171:0/3107718772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:47:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:39 smithi129 ceph-mon[62418]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-28T12:47:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:39 smithi129 ceph-mon[62418]: from='client.? 172.21.15.171:0/3107718772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:47:39.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:39 smithi106 ceph-mon[28081]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-28T12:47:39.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:39 smithi106 ceph-mon[28081]: from='client.? 172.21.15.171:0/3107718772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-28T12:47:41.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:41 smithi171 ceph-mon[63127]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-28T12:47:41.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:41 smithi129 ceph-mon[62418]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-28T12:47:41.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:41 smithi106 ceph-mon[28081]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-04-28T12:47:42.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:42 smithi171 ceph-mon[63127]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-28T12:47:42.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:42 smithi129 ceph-mon[62418]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-28T12:47:42.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:42 smithi106 ceph-mon[28081]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-28T12:47:45.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:45 smithi171 ceph-mon[63127]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-28T12:47:45.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:45 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-28T12:47:45.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:45 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:45.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:45 smithi129 ceph-mon[62418]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-28T12:47:45.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:45 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-28T12:47:45.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:45 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:45.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:45 smithi106 ceph-mon[28081]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-28T12:47:45.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:45 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-28T12:47:45.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:45 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:47:46.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:46 smithi171 ceph-mon[63127]: Deploying daemon osd.11 on smithi171 2024-04-28T12:47:46.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:46 smithi129 ceph-mon[62418]: Deploying daemon osd.11 on smithi171 2024-04-28T12:47:46.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:46 smithi106 ceph-mon[28081]: Deploying daemon osd.11 on smithi171 2024-04-28T12:47:47.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:47 smithi171 ceph-mon[63127]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:47 smithi129 ceph-mon[62418]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:47.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:47 smithi106 ceph-mon[28081]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:49 smithi171 ceph-mon[63127]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:49.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:49 smithi129 ceph-mon[62418]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:49.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:49 smithi106 ceph-mon[28081]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:51.490 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:51 smithi171 ceph-mon[63127]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:51.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:51 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:51.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:51 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:51.491 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:51 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:51 smithi129 ceph-mon[62418]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:51 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:51 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:51 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:51.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:51 smithi106 ceph-mon[28081]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:51.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:51 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:51.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:51 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:51.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:51 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:47:53.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:53 smithi171 ceph-mon[63127]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:53.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:53 smithi129 ceph-mon[62418]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:53.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:53 smithi106 ceph-mon[28081]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:54.498 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.502+0000 7fcf6cff9700 1 -- 172.21.15.171:0/2281103522 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcf70063860 con 0x7fcf5c0a3cc0 2024-04-28T12:47:54.499 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 11 on host 'smithi171' 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf5c0a3cc0 msgr2=0x7fcf5c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf60006fd0 tx=0x7fcf60008040 comp rx=0 tx=0).stop 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70109160 msgr2=0x7fcf701a2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70109160 0x7fcf701a2f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800ba10 tx=0x7fcf6800bdd0 comp rx=0 tx=0).stop 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 shutdown_connections 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf5c0a3cc0 0x7fcf5c0a6180 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf70109160 0x7fcf701a2f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf70103f00 0x7fcf7019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 --2- 172.21.15.171:0/2281103522 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf70106830 0x7fcf7019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 >> 172.21.15.171:0/2281103522 conn(0x7fcf700fd910 msgr2=0x7fcf701036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 shutdown_connections 2024-04-28T12:47:54.501 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:47:54.505+0000 7fcf757ba700 1 -- 172.21.15.171:0/2281103522 wait complete. 2024-04-28T12:47:55.252 DEBUG:teuthology.orchestra.run.smithi171:osd.11> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.11.service 2024-04-28T12:47:55.255 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-28T12:47:55.255 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-28T12:47:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:55 smithi171 ceph-mon[63127]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:55 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:55 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:55 smithi171 ceph-mon[63127]: from='osd.11 [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-28T12:47:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:55 smithi171 ceph-mon[63127]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-28T12:47:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:55 smithi129 ceph-mon[62418]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:55 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:55 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:55 smithi129 ceph-mon[62418]: from='osd.11 [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-28T12:47:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:55 smithi129 ceph-mon[62418]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-28T12:47:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:55 smithi106 ceph-mon[28081]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:55 smithi106 ceph-mon[28081]: from='osd.11 [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-28T12:47:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:55 smithi106 ceph-mon[28081]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-28T12:47:55.716 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:47:56.430 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:56.430 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:56.722 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:56 smithi106 ceph-mon[28081]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-28T12:47:56.722 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:56 smithi106 ceph-mon[28081]: from='osd.11 [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:47:56.722 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:56 smithi106 ceph-mon[28081]: osdmap e78: 12 total, 11 up, 12 in 2024-04-28T12:47:56.722 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:56.722 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:56 smithi106 ceph-mon[28081]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:47:56.722 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:56 smithi106 ceph-mon[28081]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:56.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:56.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:56.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:56 smithi171 ceph-mon[63127]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-28T12:47:56.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:56 smithi171 ceph-mon[63127]: from='osd.11 [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:47:56.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:56 smithi171 ceph-mon[63127]: osdmap e78: 12 total, 11 up, 12 in 2024-04-28T12:47:56.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:56.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:56 smithi171 ceph-mon[63127]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:47:56.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:56 smithi171 ceph-mon[63127]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:56.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:56.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:47:56.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:56 smithi129 ceph-mon[62418]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-28T12:47:56.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:56 smithi129 ceph-mon[62418]: from='osd.11 [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:47:56.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:56 smithi129 ceph-mon[62418]: osdmap e78: 12 total, 11 up, 12 in 2024-04-28T12:47:56.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:56.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:56 smithi129 ceph-mon[62418]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]: dispatch 2024-04-28T12:47:56.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:56 smithi129 ceph-mon[62418]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:57.454 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 12:47:57 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T12:47:57.134+0000 7fa18e72b700 -1 osd.11 0 waiting for initial osdmap 2024-04-28T12:47:57.454 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 12:47:57 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T12:47:57.151+0000 7fa186516700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-28T12:47:57.712 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:57 smithi106 ceph-mon[28081]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:47:57.712 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:57 smithi106 ceph-mon[28081]: osdmap e79: 12 total, 11 up, 12 in 2024-04-28T12:47:57.712 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:57.712 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:57.712 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:57 smithi106 ceph-mon[28081]: from='osd.11 ' entity='osd.11' 2024-04-28T12:47:57.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:57 smithi171 ceph-mon[63127]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:47:57.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:57 smithi171 ceph-mon[63127]: osdmap e79: 12 total, 11 up, 12 in 2024-04-28T12:47:57.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:57.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:57.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:57 smithi171 ceph-mon[63127]: from='osd.11 ' entity='osd.11' 2024-04-28T12:47:57.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:57 smithi129 ceph-mon[62418]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-04-28T12:47:57.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:57 smithi129 ceph-mon[62418]: osdmap e79: 12 total, 11 up, 12 in 2024-04-28T12:47:57.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:57.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:57.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:57 smithi129 ceph-mon[62418]: from='osd.11 ' entity='osd.11' 2024-04-28T12:47:57.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.990+0000 7f948f200700 1 -- 172.21.15.106:0/3626563576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901067c0 msgr2=0x7f9490106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:57.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.990+0000 7f948f200700 1 --2- 172.21.15.106:0/3626563576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901067c0 0x7f9490106c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f947c009990 tx=0x7f947c009ca0 comp rx=0 tx=0).stop 2024-04-28T12:47:57.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.990+0000 7f948f200700 1 -- 172.21.15.106:0/3626563576 shutdown_connections 2024-04-28T12:47:57.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.990+0000 7f948f200700 1 --2- 172.21.15.106:0/3626563576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9490107160 0x7f949010da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:57.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.990+0000 7f948f200700 1 --2- 172.21.15.106:0/3626563576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94901055c0 0x7f94901059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:57.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.990+0000 7f948f200700 1 --2- 172.21.15.106:0/3626563576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901067c0 0x7f9490106c20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:57.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.990+0000 7f948f200700 1 -- 172.21.15.106:0/3626563576 >> 172.21.15.106:0/3626563576 conn(0x7f9490100b40 msgr2=0x7f9490102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:47:57.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.990+0000 7f948f200700 1 -- 172.21.15.106:0/3626563576 shutdown_connections 2024-04-28T12:47:57.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.990+0000 7f948f200700 1 -- 172.21.15.106:0/3626563576 wait complete. 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.992+0000 7f948f200700 1 Processor -- start 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.992+0000 7f948f200700 1 -- start start 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.992+0000 7f948f200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901055c0 0x7f949019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.992+0000 7f948f200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94901067c0 0x7f949019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.992+0000 7f948f200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9490107160 0x7f94901a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.992+0000 7f948f200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9490111090 con 0x7f94901067c0 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.992+0000 7f948f200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9490110f10 con 0x7f9490107160 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.992+0000 7f948f200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9490111210 con 0x7f94901055c0 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901055c0 0x7f949019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901055c0 0x7f949019b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:46938/0 (socket says 172.21.15.106:46938) 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948de00700 1 -- 172.21.15.106:0/2131577949 learned_addr learned my addr 172.21.15.106:0/2131577949 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:47:57.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948e800700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9490107160 0x7f94901a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948d400700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94901067c0 0x7f949019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948de00700 1 -- 172.21.15.106:0/2131577949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9490107160 msgr2=0x7f94901a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948de00700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9490107160 0x7f94901a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948de00700 1 -- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94901067c0 msgr2=0x7f949019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948de00700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94901067c0 0x7f949019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948de00700 1 -- 172.21.15.106:0/2131577949 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94901a3330 con 0x7f94901055c0 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.993+0000 7f948de00700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901055c0 0x7f949019b6d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f948000e9a0 tx=0x7f948000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.994+0000 7f9486a00700 1 -- 172.21.15.106:0/2131577949 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480004220 con 0x7f94901055c0 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.994+0000 7f9486a00700 1 -- 172.21.15.106:0/2131577949 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94800043c0 con 0x7f94901055c0 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.994+0000 7f9486a00700 1 -- 172.21.15.106:0/2131577949 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480003cc0 con 0x7f94901055c0 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.994+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94901a3620 con 0x7f94901055c0 2024-04-28T12:47:57.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.994+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94901a3b10 con 0x7f94901055c0 2024-04-28T12:47:57.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.995+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f949006a110 con 0x7f94901055c0 2024-04-28T12:47:57.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.997+0000 7f9486a00700 1 -- 172.21.15.106:0/2131577949 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f948000c810 con 0x7f94901055c0 2024-04-28T12:47:57.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.998+0000 7f9486a00700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94740a3cc0 0x7f94740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:47:57.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.998+0000 7f9486a00700 1 -- 172.21.15.106:0/2131577949 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f94800cfff0 con 0x7f94901055c0 2024-04-28T12:47:57.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.998+0000 7f948d400700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94740a3cc0 0x7f94740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:47:57.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:57.999+0000 7f948d400700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94740a3cc0 0x7f94740a6180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f947c00b720 tx=0x7f947c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:47:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.002+0000 7f9486a00700 1 -- 172.21.15.106:0/2131577949 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f948009cef0 con 0x7f94901055c0 2024-04-28T12:47:58.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.238+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9490064030 con 0x7f94901055c0 2024-04-28T12:47:58.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.239+0000 7f9486a00700 1 -- 172.21.15.106:0/2131577949 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f948009c7f0 con 0x7f94901055c0 2024-04-28T12:47:58.235 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:47:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94740a3cc0 msgr2=0x7f94740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94740a3cc0 0x7f94740a6180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f947c00b720 tx=0x7f947c005fd0 comp rx=0 tx=0).stop 2024-04-28T12:47:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901055c0 msgr2=0x7f949019b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:47:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901055c0 0x7f949019b6d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f948000e9a0 tx=0x7f948000ecb0 comp rx=0 tx=0).stop 2024-04-28T12:47:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 shutdown_connections 2024-04-28T12:47:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9490107160 0x7f94901a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94740a3cc0 0x7f94740a6180 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94901055c0 0x7f949019b6d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 --2- 172.21.15.106:0/2131577949 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94901067c0 0x7f949019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:47:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 >> 172.21.15.106:0/2131577949 conn(0x7f9490100b40 msgr2=0x7f9490109ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:47:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 shutdown_connections 2024-04-28T12:47:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:47:58.242+0000 7f948f200700 1 -- 172.21.15.106:0/2131577949 wait complete. 2024-04-28T12:47:58.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:58 smithi106 ceph-mon[28081]: purged_snaps scrub starts 2024-04-28T12:47:58.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:58 smithi106 ceph-mon[28081]: purged_snaps scrub ok 2024-04-28T12:47:58.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:58.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:58 smithi106 ceph-mon[28081]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:58.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:58 smithi106 ceph-mon[28081]: osd.11 [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] boot 2024-04-28T12:47:58.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:58 smithi106 ceph-mon[28081]: osdmap e80: 12 total, 12 up, 12 in 2024-04-28T12:47:58.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:58.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:58 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2131577949' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-28T12:47:58.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:58 smithi171 ceph-mon[63127]: purged_snaps scrub starts 2024-04-28T12:47:58.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:58 smithi171 ceph-mon[63127]: purged_snaps scrub ok 2024-04-28T12:47:58.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:58.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:58 smithi171 ceph-mon[63127]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:58.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:58 smithi171 ceph-mon[63127]: osd.11 [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] boot 2024-04-28T12:47:58.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:58 smithi171 ceph-mon[63127]: osdmap e80: 12 total, 12 up, 12 in 2024-04-28T12:47:58.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:58.608 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:58 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2131577949' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-28T12:47:58.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:58 smithi129 ceph-mon[62418]: purged_snaps scrub starts 2024-04-28T12:47:58.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:58 smithi129 ceph-mon[62418]: purged_snaps scrub ok 2024-04-28T12:47:58.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:58.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:58 smithi129 ceph-mon[62418]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-28T12:47:58.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:58 smithi129 ceph-mon[62418]: osd.11 [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] boot 2024-04-28T12:47:58.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:58 smithi129 ceph-mon[62418]: osdmap e80: 12 total, 12 up, 12 in 2024-04-28T12:47:58.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-28T12:47:58.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:58 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2131577949' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-28T12:47:59.033 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1714308478,"num_in_osds":12,"osd_in_since":1714308457,"num_remapped_pgs":0} 2024-04-28T12:47:59.034 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-28T12:47:59.438 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:48:00.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: Detected new or changed devices on smithi171 2024-04-28T12:48:00.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-28T12:48:00.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:00.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:00.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:00.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: osdmap e81: 12 total, 12 up, 12 in 2024-04-28T12:48:00.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:00.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:00.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:47:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: Detected new or changed devices on smithi171 2024-04-28T12:48:00.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: osdmap e81: 12 total, 12 up, 12 in 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:00.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:47:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: Detected new or changed devices on smithi171 2024-04-28T12:48:00.370 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.370 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.370 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.370 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.370 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi171 to 4272M 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: osdmap e81: 12 total, 12 up, 12 in 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:00.371 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:47:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:01.065 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:00 smithi106 ceph-mon[28081]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:01.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:00 smithi171 ceph-mon[63127]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:01.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:00 smithi129 ceph-mon[62418]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:01.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.603+0000 7f75a2200700 1 -- 172.21.15.106:0/1734963367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f759c1034d0 msgr2=0x7f759c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:01.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.603+0000 7f75a2200700 1 --2- 172.21.15.106:0/1734963367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f759c1034d0 0x7f759c103950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f75880098f0 tx=0x7f7588009c00 comp rx=0 tx=0).stop 2024-04-28T12:48:01.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.603+0000 7f75a2200700 1 -- 172.21.15.106:0/1734963367 shutdown_connections 2024-04-28T12:48:01.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.603+0000 7f75a2200700 1 --2- 172.21.15.106:0/1734963367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c103e90 0x7f759c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:01.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.603+0000 7f75a2200700 1 --2- 172.21.15.106:0/1734963367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f759c1083b0 0x7f759c108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:01.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.603+0000 7f75a2200700 1 --2- 172.21.15.106:0/1734963367 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f759c1034d0 0x7f759c103950 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:01.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.603+0000 7f75a2200700 1 -- 172.21.15.106:0/1734963367 >> 172.21.15.106:0/1734963367 conn(0x7f759c07a580 msgr2=0x7f759c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:01.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.604+0000 7f75a2200700 1 -- 172.21.15.106:0/1734963367 shutdown_connections 2024-04-28T12:48:01.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.604+0000 7f75a2200700 1 -- 172.21.15.106:0/1734963367 wait complete. 2024-04-28T12:48:01.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.605+0000 7f75a2200700 1 Processor -- start 2024-04-28T12:48:01.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.605+0000 7f75a2200700 1 -- start start 2024-04-28T12:48:01.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f75a2200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f759c1034d0 0x7f759c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:01.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f75a2200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c103e90 0x7f759c19a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f75a2200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f759c1083b0 0x7f759c19e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f75a2200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f759c113670 con 0x7f759c103e90 2024-04-28T12:48:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f75a2200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1134f0 con 0x7f759c1083b0 2024-04-28T12:48:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f75a2200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1137f0 con 0x7f759c1034d0 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f759aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c103e90 0x7f759c19a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f759be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f759c1083b0 0x7f759c19e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f759aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c103e90 0x7f759c19a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43998/0 (socket says 172.21.15.106:43998) 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f759aa00700 1 -- 172.21.15.106:0/402579884 learned_addr learned my addr 172.21.15.106:0/402579884 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f759b400700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f759c1034d0 0x7f759c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f759aa00700 1 -- 172.21.15.106:0/402579884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f759c1034d0 msgr2=0x7f759c199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f759aa00700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f759c1034d0 0x7f759c199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f759aa00700 1 -- 172.21.15.106:0/402579884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f759c1083b0 msgr2=0x7f759c19e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.606+0000 7f759aa00700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f759c1083b0 0x7f759c19e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.607+0000 7f759aa00700 1 -- 172.21.15.106:0/402579884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c19ec00 con 0x7f759c103e90 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.607+0000 7f759aa00700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c103e90 0x7f759c19a140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7588009230 tx=0x7f758800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.607+0000 7f7593e00700 1 -- 172.21.15.106:0/402579884 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75880050c0 con 0x7f759c103e90 2024-04-28T12:48:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.607+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c19ee90 con 0x7f759c103e90 2024-04-28T12:48:01.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.607+0000 7f7593e00700 1 -- 172.21.15.106:0/402579884 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f758800b560 con 0x7f759c103e90 2024-04-28T12:48:01.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.607+0000 7f7593e00700 1 -- 172.21.15.106:0/402579884 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588003a00 con 0x7f759c103e90 2024-04-28T12:48:01.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.607+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c1a67b0 con 0x7f759c103e90 2024-04-28T12:48:01.605 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.608+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759c104b90 con 0x7f759c103e90 2024-04-28T12:48:01.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.610+0000 7f7593e00700 1 -- 172.21.15.106:0/402579884 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f758800b710 con 0x7f759c103e90 2024-04-28T12:48:01.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.612+0000 7f7593e00700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75800a3c40 0x7f75800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:01.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.612+0000 7f7593e00700 1 -- 172.21.15.106:0/402579884 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f75880d0ec0 con 0x7f759c103e90 2024-04-28T12:48:01.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.612+0000 7f759b400700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75800a3c40 0x7f75800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:01.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.613+0000 7f759b400700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75800a3c40 0x7f75800a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f758c009be0 tx=0x7f758c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:01.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.617+0000 7f7593e00700 1 -- 172.21.15.106:0/402579884 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7588013030 con 0x7f759c103e90 2024-04-28T12:48:01.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.846+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f759c06a110 con 0x7f759c103e90 2024-04-28T12:48:01.843 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.847+0000 7f7593e00700 1 -- 172.21.15.106:0/402579884 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18524 (secure 0 0 0) 0x7f7588021380 con 0x7f759c103e90 2024-04-28T12:48:01.844 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:01.844 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":81,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:47:59.467490+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:48:01.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75800a3c40 msgr2=0x7f75800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:01.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75800a3c40 0x7f75800a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f758c009be0 tx=0x7f758c009260 comp rx=0 tx=0).stop 2024-04-28T12:48:01.846 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c103e90 msgr2=0x7f759c19a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:01.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c103e90 0x7f759c19a140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7588009230 tx=0x7f758800fa70 comp rx=0 tx=0).stop 2024-04-28T12:48:01.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 shutdown_connections 2024-04-28T12:48:01.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f759c1083b0 0x7f759c19e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:01.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75800a3c40 0x7f75800a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:01.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f759c1034d0 0x7f759c199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:01.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 --2- 172.21.15.106:0/402579884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f759c103e90 0x7f759c19a140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:01.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 >> 172.21.15.106:0/402579884 conn(0x7f759c07a580 msgr2=0x7f759c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:01.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 shutdown_connections 2024-04-28T12:48:01.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:01.850+0000 7f75a2200700 1 -- 172.21.15.106:0/402579884 wait complete. 2024-04-28T12:48:02.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:01 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/402579884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:48:02.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:01 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/402579884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:48:02.264 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:01 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/402579884' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:48:02.669 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T12:48:02.669 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-28T12:48:03.099 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:48:03.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:03 smithi129 ceph-mon[62418]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:03.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:03 smithi106 ceph-mon[28081]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:03 smithi171 ceph-mon[63127]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:05.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.233+0000 7fea31a00700 1 -- 172.21.15.106:0/2332125450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea2c1083b0 msgr2=0x7fea2c108790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:05.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.233+0000 7fea31a00700 1 --2- 172.21.15.106:0/2332125450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea2c1083b0 0x7fea2c108790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009f90 tx=0x7fea1c00b330 comp rx=0 tx=0).stop 2024-04-28T12:48:05.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.233+0000 7fea31a00700 1 -- 172.21.15.106:0/2332125450 shutdown_connections 2024-04-28T12:48:05.228 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.233+0000 7fea31a00700 1 --2- 172.21.15.106:0/2332125450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea2c103e90 0x7fea2c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.233+0000 7fea31a00700 1 --2- 172.21.15.106:0/2332125450 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea2c1083b0 0x7fea2c108790 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.233+0000 7fea31a00700 1 --2- 172.21.15.106:0/2332125450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea2c1034d0 0x7fea2c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.233+0000 7fea31a00700 1 -- 172.21.15.106:0/2332125450 >> 172.21.15.106:0/2332125450 conn(0x7fea2c07a580 msgr2=0x7fea2c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:05.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.233+0000 7fea31a00700 1 -- 172.21.15.106:0/2332125450 shutdown_connections 2024-04-28T12:48:05.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.233+0000 7fea31a00700 1 -- 172.21.15.106:0/2332125450 wait complete. 2024-04-28T12:48:05.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.234+0000 7fea31a00700 1 Processor -- start 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.234+0000 7fea31a00700 1 -- start start 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea31a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea2c1034d0 0x7fea2c19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea31a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea2c103e90 0x7fea2c19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea31a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea2c1083b0 0x7fea2c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea31a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c113660 con 0x7fea2c1083b0 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea31a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1134e0 con 0x7fea2c103e90 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea31a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1137e0 con 0x7fea2c1034d0 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea22000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea2c103e90 0x7fea2c19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea22000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea2c103e90 0x7fea2c19c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:59702/0 (socket says 172.21.15.106:59702) 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea22000700 1 -- 172.21.15.106:0/3677366080 learned_addr learned my addr 172.21.15.106:0/3677366080 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:05.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea2b400700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea2c1083b0 0x7fea2c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.235+0000 7fea2aa00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea2c1034d0 0x7fea2c19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea22000700 1 -- 172.21.15.106:0/3677366080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea2c1034d0 msgr2=0x7fea2c19bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea22000700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea2c1034d0 0x7fea2c19bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea22000700 1 -- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea2c1083b0 msgr2=0x7fea2c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea22000700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea2c1083b0 0x7fea2c1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea22000700 1 -- 172.21.15.106:0/3677366080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c1a0d20 con 0x7fea2c103e90 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea22000700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea2c103e90 0x7fea2c19c2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea1400ec10 tx=0x7fea1400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea23e00700 1 -- 172.21.15.106:0/3677366080 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1400cc90 con 0x7fea2c103e90 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea23e00700 1 -- 172.21.15.106:0/3677366080 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea1400ce30 con 0x7fea2c103e90 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea23e00700 1 -- 172.21.15.106:0/3677366080 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14010430 con 0x7fea2c103e90 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c1a1010 con 0x7fea2c103e90 2024-04-28T12:48:05.233 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.236+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1a8890 con 0x7fea2c103e90 2024-04-28T12:48:05.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.237+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2c06a110 con 0x7fea2c103e90 2024-04-28T12:48:05.234 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.239+0000 7fea23e00700 1 -- 172.21.15.106:0/3677366080 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fea14004000 con 0x7fea2c103e90 2024-04-28T12:48:05.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.240+0000 7fea23e00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fea100a3cc0 0x7fea100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:05.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.241+0000 7fea23e00700 1 -- 172.21.15.106:0/3677366080 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fea140d0200 con 0x7fea2c103e90 2024-04-28T12:48:05.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.241+0000 7fea2aa00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fea100a3cc0 0x7fea100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:05.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.241+0000 7fea2aa00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fea100a3cc0 0x7fea100a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b6c0 tx=0x7fea1c0096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:05.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.245+0000 7fea23e00700 1 -- 172.21.15.106:0/3677366080 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fea1409cfe0 con 0x7fea2c103e90 2024-04-28T12:48:05.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:05 smithi106 ceph-mon[28081]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.468+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea2c1a8a90 con 0x7fea2c103e90 2024-04-28T12:48:05.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.469+0000 7fea23e00700 1 -- 172.21.15.106:0/3677366080 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fea1409c8e0 con 0x7fea2c103e90 2024-04-28T12:48:05.464 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T12:48:05.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.472+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fea100a3cc0 msgr2=0x7fea100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:05.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.472+0000 7fea31a00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fea100a3cc0 0x7fea100a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b6c0 tx=0x7fea1c0096c0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.472+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea2c103e90 msgr2=0x7fea2c19c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:05.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.472+0000 7fea31a00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea2c103e90 0x7fea2c19c2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fea1400ec10 tx=0x7fea1400ef20 comp rx=0 tx=0).stop 2024-04-28T12:48:05.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.473+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 shutdown_connections 2024-04-28T12:48:05.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.473+0000 7fea31a00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea2c1083b0 0x7fea2c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.473+0000 7fea31a00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fea100a3cc0 0x7fea100a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.473+0000 7fea31a00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea2c1034d0 0x7fea2c19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.473+0000 7fea31a00700 1 --2- 172.21.15.106:0/3677366080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea2c103e90 0x7fea2c19c2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:05.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.473+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 >> 172.21.15.106:0/3677366080 conn(0x7fea2c07a580 msgr2=0x7fea2c101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:05.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.473+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 shutdown_connections 2024-04-28T12:48:05.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:05.473+0000 7fea31a00700 1 -- 172.21.15.106:0/3677366080 wait complete. 2024-04-28T12:48:05.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:05 smithi171 ceph-mon[63127]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:05 smithi129 ceph-mon[62418]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:06.247 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:06 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3677366080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:48:06.316 INFO:tasks.cephadm:Adding mds.a on smithi106 2024-04-28T12:48:06.316 INFO:tasks.cephadm:Adding mds.d on smithi106 2024-04-28T12:48:06.316 INFO:tasks.cephadm:Adding mds.g on smithi106 2024-04-28T12:48:06.316 INFO:tasks.cephadm:Adding mds.j on smithi106 2024-04-28T12:48:06.316 INFO:tasks.cephadm:Adding mds.b on smithi129 2024-04-28T12:48:06.316 INFO:tasks.cephadm:Adding mds.e on smithi129 2024-04-28T12:48:06.317 INFO:tasks.cephadm:Adding mds.h on smithi129 2024-04-28T12:48:06.317 INFO:tasks.cephadm:Adding mds.k on smithi129 2024-04-28T12:48:06.317 INFO:tasks.cephadm:Adding mds.c on smithi171 2024-04-28T12:48:06.317 INFO:tasks.cephadm:Adding mds.f on smithi171 2024-04-28T12:48:06.317 INFO:tasks.cephadm:Adding mds.i on smithi171 2024-04-28T12:48:06.317 INFO:tasks.cephadm:Adding mds.l on smithi171 2024-04-28T12:48:06.317 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi106=a;smithi106=d;smithi106=g;smithi106=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l' 2024-04-28T12:48:06.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:06 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3677366080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:48:06.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:06 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3677366080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:48:06.566 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.c/config 2024-04-28T12:48:07.477 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:07 smithi106 ceph-mon[28081]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:07.494 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:07 smithi171 ceph-mon[63127]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:07 smithi129 ceph-mon[62418]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:08.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.230+0000 7f4dd08fa700 1 -- 172.21.15.171:0/187374160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc1034b0 msgr2=0x7f4dcc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:08.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.230+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/187374160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc1034b0 0x7f4dcc1038d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00b0a0 tx=0x7f4dbc00b3b0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.230+0000 7f4dd08fa700 1 -- 172.21.15.171:0/187374160 shutdown_connections 2024-04-28T12:48:08.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.230+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/187374160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dcc105050 0x7f4dcc10b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.230+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/187374160 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc1034b0 0x7f4dcc1038d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.230+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/187374160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dcc1046b0 0x7f4dcc104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.226 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.230+0000 7f4dd08fa700 1 -- 172.21.15.171:0/187374160 >> 172.21.15.171:0/187374160 conn(0x7f4dcc0fea70 msgr2=0x7f4dcc100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:08.227 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.230+0000 7f4dd08fa700 1 -- 172.21.15.171:0/187374160 shutdown_connections 2024-04-28T12:48:08.227 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.230+0000 7f4dd08fa700 1 -- 172.21.15.171:0/187374160 wait complete. 2024-04-28T12:48:08.227 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dd08fa700 1 Processor -- start 2024-04-28T12:48:08.228 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dd08fa700 1 -- start start 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dd08fa700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dcc1034b0 0x7f4dcc199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dd08fa700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dcc1046b0 0x7f4dcc199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dd08fa700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc105050 0x7f4dcc1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dd08fa700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc10f020 con 0x7f4dcc1046b0 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dd08fa700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc10eea0 con 0x7f4dcc1034b0 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dd08fa700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc10f1a0 con 0x7f4dcc105050 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dcad9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc105050 0x7f4dcc1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dcad9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc105050 0x7f4dcc1a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:41484/0 (socket says 172.21.15.171:41484) 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dcad9d700 1 -- 172.21.15.171:0/1631651853 learned_addr learned my addr 172.21.15.171:0/1631651853 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dca59c700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dcc1034b0 0x7f4dcc199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dcad9d700 1 -- 172.21.15.171:0/1631651853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dcc1034b0 msgr2=0x7f4dcc199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:08.229 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dcad9d700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dcc1034b0 0x7f4dcc199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.230 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dcad9d700 1 -- 172.21.15.171:0/1631651853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dcc1046b0 msgr2=0x7f4dcc199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:08.230 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dcad9d700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dcc1046b0 0x7f4dcc199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.230 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.231+0000 7f4dcad9d700 1 -- 172.21.15.171:0/1631651853 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc1a1390 con 0x7f4dcc105050 2024-04-28T12:48:08.230 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.232+0000 7f4dcad9d700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc105050 0x7f4dcc1a0cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400b580 tx=0x7f4dc400b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:08.230 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.232+0000 7f4dc3fff700 1 -- 172.21.15.171:0/1631651853 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4015020 con 0x7f4dcc105050 2024-04-28T12:48:08.230 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.232+0000 7f4dc3fff700 1 -- 172.21.15.171:0/1631651853 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4011ce0 con 0x7f4dcc105050 2024-04-28T12:48:08.230 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.232+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc1a1680 con 0x7f4dcc105050 2024-04-28T12:48:08.231 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.232+0000 7f4dc3fff700 1 -- 172.21.15.171:0/1631651853 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4010dd0 con 0x7f4dcc105050 2024-04-28T12:48:08.231 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.232+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc1a1c40 con 0x7f4dcc105050 2024-04-28T12:48:08.231 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.233+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dcc06a110 con 0x7f4dcc105050 2024-04-28T12:48:08.232 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.234+0000 7f4dc3fff700 1 -- 172.21.15.171:0/1631651853 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4dc4018020 con 0x7f4dcc105050 2024-04-28T12:48:08.232 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.234+0000 7f4dc3fff700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4db80a3c40 0x7f4db80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:08.233 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.234+0000 7f4dc3fff700 1 -- 172.21.15.171:0/1631651853 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4dc40cf660 con 0x7f4dcc105050 2024-04-28T12:48:08.233 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.235+0000 7f4dca59c700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4db80a3c40 0x7f4db80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:08.233 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.236+0000 7f4dca59c700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4db80a3c40 0x7f4db80a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009fd0 tx=0x7f4dbc006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:08.234 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.239+0000 7f4dc3fff700 1 -- 172.21.15.171:0/1631651853 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4dc401f020 con 0x7f4dcc105050 2024-04-28T12:48:08.443 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.447+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi106=a;smithi106=d;smithi106=g;smithi106=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}) -- 0x7f4dcc063a60 con 0x7f4db80a3c40 2024-04-28T12:48:08.460 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.464+0000 7f4dc3fff700 1 -- 172.21.15.171:0/1631651853 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4dcc063a60 con 0x7f4db80a3c40 2024-04-28T12:48:08.460 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mds.all update... 2024-04-28T12:48:08.462 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4db80a3c40 msgr2=0x7f4db80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:08.462 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4db80a3c40 0x7f4db80a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009fd0 tx=0x7f4dbc006000 comp rx=0 tx=0).stop 2024-04-28T12:48:08.462 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc105050 msgr2=0x7f4dcc1a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:08.462 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc105050 0x7f4dcc1a0cf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400b580 tx=0x7f4dc400b890 comp rx=0 tx=0).stop 2024-04-28T12:48:08.462 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 shutdown_connections 2024-04-28T12:48:08.463 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4db80a3c40 0x7f4db80a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.463 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4dcc105050 0x7f4dcc1a0cf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.463 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dcc1034b0 0x7f4dcc199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.463 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 --2- 172.21.15.171:0/1631651853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dcc1046b0 0x7f4dcc199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:08.463 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 >> 172.21.15.171:0/1631651853 conn(0x7f4dcc0fea70 msgr2=0x7f4dcc107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:08.463 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 shutdown_connections 2024-04-28T12:48:08.463 INFO:teuthology.orchestra.run.smithi171.stderr:2024-04-28T12:48:08.466+0000 7f4dd08fa700 1 -- 172.21.15.171:0/1631651853 wait complete. 2024-04-28T12:48:09.105 DEBUG:teuthology.orchestra.run.smithi106:mds.a> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.a.service 2024-04-28T12:48:09.112 DEBUG:teuthology.orchestra.run.smithi106:mds.d> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.d.service 2024-04-28T12:48:09.158 DEBUG:teuthology.orchestra.run.smithi106:mds.g> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.g.service 2024-04-28T12:48:09.203 DEBUG:teuthology.orchestra.run.smithi106:mds.j> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.j.service 2024-04-28T12:48:09.249 DEBUG:teuthology.orchestra.run.smithi129:mds.b> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.b.service 2024-04-28T12:48:09.252 DEBUG:teuthology.orchestra.run.smithi129:mds.e> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.e.service 2024-04-28T12:48:09.295 DEBUG:teuthology.orchestra.run.smithi129:mds.h> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.h.service 2024-04-28T12:48:09.338 DEBUG:teuthology.orchestra.run.smithi129:mds.k> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.k.service 2024-04-28T12:48:09.358 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:09 smithi129 ceph-mon[62418]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:09.358 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:09 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:09.358 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:09 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:09.359 DEBUG:teuthology.orchestra.run.smithi171:mds.c> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.c.service 2024-04-28T12:48:09.361 DEBUG:teuthology.orchestra.run.smithi171:mds.f> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.f.service 2024-04-28T12:48:09.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:09 smithi171 ceph-mon[63127]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:09.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:09 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:09.381 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:09 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:09.382 DEBUG:teuthology.orchestra.run.smithi171:mds.i> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.i.service 2024-04-28T12:48:09.425 DEBUG:teuthology.orchestra.run.smithi171:mds.l> sudo journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.l.service 2024-04-28T12:48:09.468 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-28T12:48:09.469 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:48:09.540 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:09 smithi106 ceph-mon[28081]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:09.540 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:09 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:09.540 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:09 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.692+0000 7f36b4200640 1 -- 172.21.15.106:0/2150653357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36ac076970 msgr2=0x7f36ac074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.692+0000 7f36b4200640 1 --2- 172.21.15.106:0/2150653357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36ac076970 0x7f36ac074dc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009900 tx=0x7f36a002a410 comp rx=0 tx=0).stop 2024-04-28T12:48:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.693+0000 7f36b4200640 1 -- 172.21.15.106:0/2150653357 shutdown_connections 2024-04-28T12:48:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.693+0000 7f36b4200640 1 --2- 172.21.15.106:0/2150653357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36ac10a830 0x7f36ac10ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.693+0000 7f36b4200640 1 --2- 172.21.15.106:0/2150653357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36ac075390 0x7f36ac075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.693+0000 7f36b4200640 1 --2- 172.21.15.106:0/2150653357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36ac076970 0x7f36ac074dc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.693+0000 7f36b4200640 1 -- 172.21.15.106:0/2150653357 >> 172.21.15.106:0/2150653357 conn(0x7f36ac0fdee0 msgr2=0x7f36ac100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:09.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.693+0000 7f36b4200640 1 -- 172.21.15.106:0/2150653357 shutdown_connections 2024-04-28T12:48:09.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.693+0000 7f36b4200640 1 -- 172.21.15.106:0/2150653357 wait complete. 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.694+0000 7f36b4200640 1 Processor -- start 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.694+0000 7f36b4200640 1 -- start start 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.694+0000 7f36b4200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36ac075390 0x7f36ac19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.694+0000 7f36b4200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36ac076970 0x7f36ac19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.694+0000 7f36b4200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36ac10a830 0x7f36ac1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.694+0000 7f36b4200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac110660 con 0x7f36ac10a830 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.694+0000 7f36b4200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac1104e0 con 0x7f36ac076970 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.694+0000 7f36b4200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac1107e0 con 0x7f36ac075390 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b2200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36ac10a830 0x7f36ac1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:09.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b1800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36ac075390 0x7f36ac19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b2200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36ac10a830 0x7f36ac1a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47632/0 (socket says 172.21.15.106:47632) 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b1800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36ac075390 0x7f36ac19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:58758/0 (socket says 172.21.15.106:58758) 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b2200640 1 -- 172.21.15.106:0/503808630 learned_addr learned my addr 172.21.15.106:0/503808630 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b0e00640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36ac076970 0x7f36ac19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b1800640 1 -- 172.21.15.106:0/503808630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36ac076970 msgr2=0x7f36ac19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b1800640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36ac076970 0x7f36ac19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b1800640 1 -- 172.21.15.106:0/503808630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36ac10a830 msgr2=0x7f36ac1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b1800640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36ac10a830 0x7f36ac1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b1800640 1 -- 172.21.15.106:0/503808630 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac1a2980 con 0x7f36ac075390 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.695+0000 7f36b1800640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36ac075390 0x7f36ac19ad60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36a0006010 tx=0x7f36a00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.696+0000 7f36aa000640 1 -- 172.21.15.106:0/503808630 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a003f030 con 0x7f36ac075390 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.696+0000 7f36aa000640 1 -- 172.21.15.106:0/503808630 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36a0031910 con 0x7f36ac075390 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.696+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ac1a2c10 con 0x7f36ac075390 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.696+0000 7f36aa000640 1 -- 172.21.15.106:0/503808630 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0035070 con 0x7f36ac075390 2024-04-28T12:48:09.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.696+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ac1a3150 con 0x7f36ac075390 2024-04-28T12:48:09.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.697+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ac109560 con 0x7f36ac075390 2024-04-28T12:48:09.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.699+0000 7f36aa000640 1 -- 172.21.15.106:0/503808630 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f36a0042030 con 0x7f36ac075390 2024-04-28T12:48:09.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.700+0000 7f36aa000640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f36940a3c30 0x7f36940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:09.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.700+0000 7f36aa000640 1 -- 172.21.15.106:0/503808630 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f36a00f03f0 con 0x7f36ac075390 2024-04-28T12:48:09.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.700+0000 7f36b0e00640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f36940a3c30 0x7f36940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:09.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.701+0000 7f36b0e00640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f36940a3c30 0x7f36940a60f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36ac19c270 tx=0x7f3698005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:09.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.704+0000 7f36aa000640 1 -- 172.21.15.106:0/503808630 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f36a00bcce0 con 0x7f36ac075390 2024-04-28T12:48:09.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.906+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36ac02dca0 con 0x7f36ac075390 2024-04-28T12:48:09.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.908+0000 7f36aa000640 1 -- 172.21.15.106:0/503808630 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18524 (secure 0 0 0) 0x7f36a00c0fe0 con 0x7f36ac075390 2024-04-28T12:48:09.904 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:09.904 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":81,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:47:59.467490+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:48:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.910+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f36940a3c30 msgr2=0x7f36940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.910+0000 7f36b4200640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f36940a3c30 0x7f36940a60f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36ac19c270 tx=0x7f3698005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.910+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36ac075390 msgr2=0x7f36ac19ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:09.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.910+0000 7f36b4200640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36ac075390 0x7f36ac19ad60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36a0006010 tx=0x7f36a00042f0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.910+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 shutdown_connections 2024-04-28T12:48:09.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.911+0000 7f36b4200640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f36940a3c30 0x7f36940a60f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.911+0000 7f36b4200640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36ac10a830 0x7f36ac1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.911+0000 7f36b4200640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36ac076970 0x7f36ac19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.911+0000 7f36b4200640 1 --2- 172.21.15.106:0/503808630 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36ac075390 0x7f36ac19ad60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:09.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.911+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 >> 172.21.15.106:0/503808630 conn(0x7f36ac0fdee0 msgr2=0x7f36ac106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:09.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.911+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 shutdown_connections 2024-04-28T12:48:09.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:09.911+0000 7f36b4200640 1 -- 172.21.15.106:0/503808630 wait complete. 2024-04-28T12:48:09.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T12:48:09.927 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:48:10.137 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.141+0000 7f2223e00640 1 -- 172.21.15.106:0/2163016432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2224109fc0 msgr2=0x7f222410a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.141+0000 7f2223e00640 1 --2- 172.21.15.106:0/2163016432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2224109fc0 0x7f222410a440 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2210009900 tx=0x7f221002a410 comp rx=0 tx=0).stop 2024-04-28T12:48:10.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.142+0000 7f2223e00640 1 -- 172.21.15.106:0/2163016432 shutdown_connections 2024-04-28T12:48:10.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.142+0000 7f2223e00640 1 --2- 172.21.15.106:0/2163016432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f222410a980 0x7f22241112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.142+0000 7f2223e00640 1 --2- 172.21.15.106:0/2163016432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2224109fc0 0x7f222410a440 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.142+0000 7f2223e00640 1 --2- 172.21.15.106:0/2163016432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2224108e70 0x7f2224109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.142+0000 7f2223e00640 1 -- 172.21.15.106:0/2163016432 >> 172.21.15.106:0/2163016432 conn(0x7f2224077e00 msgr2=0x7f222407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:10.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.142+0000 7f2223e00640 1 -- 172.21.15.106:0/2163016432 shutdown_connections 2024-04-28T12:48:10.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.142+0000 7f2223e00640 1 -- 172.21.15.106:0/2163016432 wait complete. 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.143+0000 7f2223e00640 1 Processor -- start 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.143+0000 7f2223e00640 1 -- start start 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.143+0000 7f2223e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2224108e70 0x7f222419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.143+0000 7f2223e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2224109fc0 0x7f222419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.143+0000 7f2223e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f222410a980 0x7f22241a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.143+0000 7f2223e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22241149d0 con 0x7f2224109fc0 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.143+0000 7f2223e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2224114850 con 0x7f2224108e70 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.143+0000 7f2223e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2224114b50 con 0x7f222410a980 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.143+0000 7f2222000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2224109fc0 0x7f222419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2223400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f222410a980 0x7f22241a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2222a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2224108e70 0x7f222419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2223400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f222410a980 0x7f22241a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:58778/0 (socket says 172.21.15.106:58778) 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2223400640 1 -- 172.21.15.106:0/3753146901 learned_addr learned my addr 172.21.15.106:0/3753146901 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:10.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2222000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2224109fc0 0x7f222419f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47640/0 (socket says 172.21.15.106:47640) 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2223400640 1 -- 172.21.15.106:0/3753146901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2224108e70 msgr2=0x7f222419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2223400640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2224108e70 0x7f222419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2223400640 1 -- 172.21.15.106:0/3753146901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2224109fc0 msgr2=0x7f222419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2223400640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2224109fc0 0x7f222419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2223400640 1 -- 172.21.15.106:0/3753146901 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22241a6c90 con 0x7f222410a980 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2222a00640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2224108e70 0x7f222419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f2223400640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f222410a980 0x7f22241a6620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f221c00eb10 tx=0x7f221c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.144+0000 7f221b400640 1 -- 172.21.15.106:0/3753146901 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c004b20 con 0x7f222410a980 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.145+0000 7f221b400640 1 -- 172.21.15.106:0/3753146901 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f221c004cc0 con 0x7f222410a980 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.145+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22241a6f80 con 0x7f222410a980 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.145+0000 7f221b400640 1 -- 172.21.15.106:0/3753146901 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c010690 con 0x7f222410a980 2024-04-28T12:48:10.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.145+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22241a7440 con 0x7f222410a980 2024-04-28T12:48:10.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.146+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2224110200 con 0x7f222410a980 2024-04-28T12:48:10.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.148+0000 7f221b400640 1 -- 172.21.15.106:0/3753146901 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f221c005020 con 0x7f222410a980 2024-04-28T12:48:10.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.149+0000 7f221b400640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22080a3bb0 0x7f22080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:10.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.149+0000 7f221b400640 1 -- 172.21.15.106:0/3753146901 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f221c0d0530 con 0x7f222410a980 2024-04-28T12:48:10.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.149+0000 7f2222a00640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22080a3bb0 0x7f22080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:10.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.150+0000 7f2222a00640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22080a3bb0 0x7f22080a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2214009fc0 tx=0x7f2214005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:10.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.153+0000 7f221b400640 1 -- 172.21.15.106:0/3753146901 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f221c09d310 con 0x7f222410a980 2024-04-28T12:48:10.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:10 smithi106 ceph-mon[28081]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi106=a;smithi106=d;smithi106=g;smithi106=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:48:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:10 smithi106 ceph-mon[28081]: Saving service mds.all spec with placement smithi106=a;smithi106=d;smithi106=g;smithi106=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-28T12:48:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:10.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:10 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/503808630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:48:10.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.357+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2224063530 con 0x7f222410a980 2024-04-28T12:48:10.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.358+0000 7f221b400640 1 -- 172.21.15.106:0/3753146901 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f221c09cc10 con 0x7f222410a980 2024-04-28T12:48:10.354 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T12:48:10.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.360+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22080a3bb0 msgr2=0x7f22080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.360+0000 7f2223e00640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22080a3bb0 0x7f22080a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2214009fc0 tx=0x7f2214005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:10.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.360+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f222410a980 msgr2=0x7f22241a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.360+0000 7f2223e00640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f222410a980 0x7f22241a6620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f221c00eb10 tx=0x7f221c00c2d0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.361+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 shutdown_connections 2024-04-28T12:48:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.361+0000 7f2223e00640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f22080a3bb0 0x7f22080a6070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.361+0000 7f2223e00640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2224109fc0 0x7f222419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.361+0000 7f2223e00640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f222410a980 0x7f22241a6620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.361+0000 7f2223e00640 1 --2- 172.21.15.106:0/3753146901 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2224108e70 0x7f222419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.361+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 >> 172.21.15.106:0/3753146901 conn(0x7f2224077e00 msgr2=0x7f222410d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.361+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 shutdown_connections 2024-04-28T12:48:10.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.361+0000 7f2223e00640 1 -- 172.21.15.106:0/3753146901 wait complete. 2024-04-28T12:48:10.376 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:10.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:10 smithi171 ceph-mon[63127]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi106=a;smithi106=d;smithi106=g;smithi106=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:48:10.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:10 smithi171 ceph-mon[63127]: Saving service mds.all spec with placement smithi106=a;smithi106=d;smithi106=g;smithi106=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-28T12:48:10.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:10.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:10.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:10.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:10.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:10.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:10.447 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:10 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/503808630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:48:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:10 smithi129 ceph-mon[62418]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi106=a;smithi106=d;smithi106=g;smithi106=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:48:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:10 smithi129 ceph-mon[62418]: Saving service mds.all spec with placement smithi106=a;smithi106=d;smithi106=g;smithi106=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-28T12:48:10.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:10.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:10.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:10.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:10.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:10.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:10.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:10 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/503808630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:48:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.629+0000 7f944d200640 1 -- 172.21.15.106:0/1024561646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9448104960 msgr2=0x7f9448104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.629+0000 7f944d200640 1 --2- 172.21.15.106:0/1024561646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9448104960 0x7f9448104d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9438009900 tx=0x7f943802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.629+0000 7f944d200640 1 -- 172.21.15.106:0/1024561646 shutdown_connections 2024-04-28T12:48:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.629+0000 7f944d200640 1 --2- 172.21.15.106:0/1024561646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9448106510 0x7f944810cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.629+0000 7f944d200640 1 --2- 172.21.15.106:0/1024561646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9448105b50 0x7f9448105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.629+0000 7f944d200640 1 --2- 172.21.15.106:0/1024561646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9448104960 0x7f9448104d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.629+0000 7f944d200640 1 -- 172.21.15.106:0/1024561646 >> 172.21.15.106:0/1024561646 conn(0x7f94480ffeb0 msgr2=0x7f9448102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.629+0000 7f944d200640 1 -- 172.21.15.106:0/1024561646 shutdown_connections 2024-04-28T12:48:10.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.629+0000 7f944d200640 1 -- 172.21.15.106:0/1024561646 wait complete. 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.630+0000 7f944d200640 1 Processor -- start 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.630+0000 7f944d200640 1 -- start start 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.630+0000 7f944d200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9448104960 0x7f944819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f944d200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9448105b50 0x7f944819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f944d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9448106510 0x7f94481a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f944d200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f94481104d0 con 0x7f9448104960 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f944d200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9448110350 con 0x7f9448105b50 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f944d200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9448110650 con 0x7f9448106510 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9447400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9448104960 0x7f944819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9447400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9448104960 0x7f944819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47648/0 (socket says 172.21.15.106:47648) 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9447400640 1 -- 172.21.15.106:0/3986753931 learned_addr learned my addr 172.21.15.106:0/3986753931 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:10.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9446a00640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9448105b50 0x7f944819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9447e00640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9448106510 0x7f94481a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9447400640 1 -- 172.21.15.106:0/3986753931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9448106510 msgr2=0x7f94481a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9447400640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9448106510 0x7f94481a2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9447400640 1 -- 172.21.15.106:0/3986753931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9448105b50 msgr2=0x7f944819b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9447400640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9448105b50 0x7f944819b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.631+0000 7f9447400640 1 -- 172.21.15.106:0/3986753931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94481a27f0 con 0x7f9448104960 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.632+0000 7f9447400640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9448104960 0x7f944819abd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9438031500 tx=0x7f9438003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.632+0000 7f943fe00640 1 -- 172.21.15.106:0/3986753931 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943803f070 con 0x7f9448104960 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.632+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94481a2a80 con 0x7f9448104960 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.632+0000 7f943fe00640 1 -- 172.21.15.106:0/3986753931 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9438031c50 con 0x7f9448104960 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.632+0000 7f943fe00640 1 -- 172.21.15.106:0/3986753931 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438035030 con 0x7f9448104960 2024-04-28T12:48:10.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.632+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94481a2fc0 con 0x7f9448104960 2024-04-28T12:48:10.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.633+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944810bcf0 con 0x7f9448104960 2024-04-28T12:48:10.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.635+0000 7f943fe00640 1 -- 172.21.15.106:0/3986753931 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9438042070 con 0x7f9448104960 2024-04-28T12:48:10.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.637+0000 7f943fe00640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f942c0a3bb0 0x7f942c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:10.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.637+0000 7f943fe00640 1 -- 172.21.15.106:0/3986753931 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f94380f06e0 con 0x7f9448104960 2024-04-28T12:48:10.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.637+0000 7f9446a00640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f942c0a3bb0 0x7f942c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:10.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.638+0000 7f9446a00640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f942c0a3bb0 0x7f942c0a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f944819c0e0 tx=0x7f9434005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:10.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.642+0000 7f943fe00640 1 -- 172.21.15.106:0/3986753931 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f94380bce70 con 0x7f9448104960 2024-04-28T12:48:10.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.878+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9448063530 con 0x7f9448104960 2024-04-28T12:48:10.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.879+0000 7f943fe00640 1 -- 172.21.15.106:0/3986753931 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f94380c1170 con 0x7f9448104960 2024-04-28T12:48:10.875 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:10.875 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-28T12:48:10.875 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 1 2024-04-28T12:48:10.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.881+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f942c0a3bb0 msgr2=0x7f942c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.881+0000 7f944d200640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f942c0a3bb0 0x7f942c0a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f944819c0e0 tx=0x7f9434005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.881+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9448104960 msgr2=0x7f944819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.881+0000 7f944d200640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9448104960 0x7f944819abd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9438031500 tx=0x7f9438003c10 comp rx=0 tx=0).stop 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.881+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 shutdown_connections 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.881+0000 7f944d200640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f942c0a3bb0 0x7f942c0a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.881+0000 7f944d200640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9448106510 0x7f94481a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.881+0000 7f944d200640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9448105b50 0x7f944819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.881+0000 7f944d200640 1 --2- 172.21.15.106:0/3986753931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9448104960 0x7f944819abd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.882+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 >> 172.21.15.106:0/3986753931 conn(0x7f94480ffeb0 msgr2=0x7f9448108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.882+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 shutdown_connections 2024-04-28T12:48:10.878 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:10.882+0000 7f944d200640 1 -- 172.21.15.106:0/3986753931 wait complete. 2024-04-28T12:48:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:11 smithi106 ceph-mon[28081]: Deploying daemon mds.l on smithi171 2024-04-28T12:48:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:11 smithi106 ceph-mon[28081]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:11.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:11 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3753146901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:48:11.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:11 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3986753931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:11.426 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:11 smithi171 ceph-mon[63127]: Deploying daemon mds.l on smithi171 2024-04-28T12:48:11.426 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:11 smithi171 ceph-mon[63127]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:11.426 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:11 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3753146901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:48:11.427 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:11 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3986753931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:11.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:11 smithi129 ceph-mon[62418]: Deploying daemon mds.l on smithi171 2024-04-28T12:48:11.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:11 smithi129 ceph-mon[62418]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:11.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:11 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3753146901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:48:11.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:11 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3986753931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:12.063 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 12:48:12 smithi171 podman[92464]: 2024-04-28 12:48:11.925311289 +0000 UTC m=+0.018367545 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:12.063 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 12:48:12 smithi171 podman[92464]: 2024-04-28 12:48:12.067392844 +0000 UTC m=+0.160449094 container create 758fa61b1c1eef28a963fd68ea50093aebcdd115b52cb7a045888b233c23bd9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-28T12:48:12.316 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 12:48:12 smithi171 podman[92464]: 2024-04-28 12:48:12.295205872 +0000 UTC m=+0.388262125 container init 758fa61b1c1eef28a963fd68ea50093aebcdd115b52cb7a045888b233c23bd9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-28T12:48:12.316 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 12:48:12 smithi171 podman[92464]: 2024-04-28 12:48:12.297866006 +0000 UTC m=+0.390922266 container start 758fa61b1c1eef28a963fd68ea50093aebcdd115b52cb7a045888b233c23bd9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-28T12:48:12.745 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 12:48:12 smithi171 ceph-mds[92483]: -- 172.21.15.171:0/3558751745 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55722adefa40 con 0x55722adc2800 2024-04-28T12:48:12.745 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 12:48:12 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l[92479]: starting mds.l at 2024-04-28T12:48:12.745 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 12:48:12 smithi171 bash[92464]: 758fa61b1c1eef28a963fd68ea50093aebcdd115b52cb7a045888b233c23bd9a 2024-04-28T12:48:12.745 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 12:48:12 smithi171 systemd[1]: Started Ceph mds.l for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:12.897 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.114+0000 7fd78be00640 1 -- 172.21.15.106:0/400183057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c10ed40 msgr2=0x7fd78c111130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.114+0000 7fd78be00640 1 --2- 172.21.15.106:0/400183057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c10ed40 0x7fd78c111130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd784009900 tx=0x7fd78402a410 comp rx=0 tx=0).stop 2024-04-28T12:48:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.115+0000 7fd78be00640 1 -- 172.21.15.106:0/400183057 shutdown_connections 2024-04-28T12:48:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.115+0000 7fd78be00640 1 --2- 172.21.15.106:0/400183057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c075390 0x7fd78c075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.115+0000 7fd78be00640 1 --2- 172.21.15.106:0/400183057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c10ed40 0x7fd78c111130 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.115+0000 7fd78be00640 1 --2- 172.21.15.106:0/400183057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd78c076970 0x7fd78c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.115+0000 7fd78be00640 1 -- 172.21.15.106:0/400183057 >> 172.21.15.106:0/400183057 conn(0x7fd78c0fdcc0 msgr2=0x7fd78c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.115+0000 7fd78be00640 1 -- 172.21.15.106:0/400183057 shutdown_connections 2024-04-28T12:48:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.115+0000 7fd78be00640 1 -- 172.21.15.106:0/400183057 wait complete. 2024-04-28T12:48:13.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.116+0000 7fd78be00640 1 Processor -- start 2024-04-28T12:48:13.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.116+0000 7fd78be00640 1 -- start start 2024-04-28T12:48:13.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.116+0000 7fd78be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c075390 0x7fd78c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:13.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.116+0000 7fd78be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c076970 0x7fd78c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:13.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd78c10ed40 0x7fd78c1a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c114870 con 0x7fd78c076970 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78be00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1146f0 con 0x7fd78c10ed40 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd78c1149f0 con 0x7fd78c075390 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c075390 0x7fd78c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c075390 0x7fd78c19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:58816/0 (socket says 172.21.15.106:58816) 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78aa00640 1 -- 172.21.15.106:0/2074196541 learned_addr learned my addr 172.21.15.106:0/2074196541 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78b400640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd78c10ed40 0x7fd78c1a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78a000640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c076970 0x7fd78c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78aa00640 1 -- 172.21.15.106:0/2074196541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd78c10ed40 msgr2=0x7fd78c1a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78aa00640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd78c10ed40 0x7fd78c1a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78aa00640 1 -- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c076970 msgr2=0x7fd78c19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78aa00640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c076970 0x7fd78c19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78aa00640 1 -- 172.21.15.106:0/2074196541 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c1a6b20 con 0x7fd78c075390 2024-04-28T12:48:13.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78b400640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd78c10ed40 0x7fd78c1a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:13.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.117+0000 7fd78a000640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c076970 0x7fd78c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:13.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.118+0000 7fd78aa00640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c075390 0x7fd78c19ef00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00ebc0 tx=0x7fd77c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:13.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.118+0000 7fd783400640 1 -- 172.21.15.106:0/2074196541 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c010630 con 0x7fd78c075390 2024-04-28T12:48:13.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.118+0000 7fd783400640 1 -- 172.21.15.106:0/2074196541 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd77c010c80 con 0x7fd78c075390 2024-04-28T12:48:13.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.118+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78c1a6e10 con 0x7fd78c075390 2024-04-28T12:48:13.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.118+0000 7fd783400640 1 -- 172.21.15.106:0/2074196541 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c018b60 con 0x7fd78c075390 2024-04-28T12:48:13.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.118+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd78c1a7270 con 0x7fd78c075390 2024-04-28T12:48:13.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.119+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd78c104f10 con 0x7fd78c075390 2024-04-28T12:48:13.117 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.121+0000 7fd783400640 1 -- 172.21.15.106:0/2074196541 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd77c018d00 con 0x7fd78c075390 2024-04-28T12:48:13.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.123+0000 7fd783400640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7700a3bb0 0x7fd7700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:13.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.123+0000 7fd783400640 1 -- 172.21.15.106:0/2074196541 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd77c0d05c0 con 0x7fd78c075390 2024-04-28T12:48:13.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.123+0000 7fd78a000640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7700a3bb0 0x7fd7700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:13.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.124+0000 7fd78a000640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7700a3bb0 0x7fd7700a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd78c1a0410 tx=0x7fd778005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:13.123 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.127+0000 7fd783400640 1 -- 172.21.15.106:0/2074196541 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd77c0d07c0 con 0x7fd78c075390 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.171:6832/1724808557,v1:172.21.15.171:6833/1724808557] up:boot 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: fsmap 1 up:standby 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:13.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.373+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd78c02dad0 con 0x7fd78c075390 2024-04-28T12:48:13.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.374+0000 7fd783400640 1 -- 172.21.15.106:0/2074196541 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fd77c09cd60 con 0x7fd78c075390 2024-04-28T12:48:13.370 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:13.370 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-28T12:48:13.371 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 2 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7700a3bb0 msgr2=0x7fd7700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7700a3bb0 0x7fd7700a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd78c1a0410 tx=0x7fd778005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c075390 msgr2=0x7fd78c19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c075390 0x7fd78c19ef00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00ebc0 tx=0x7fd77c00c6a0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 shutdown_connections 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7700a3bb0 0x7fd7700a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd78c076970 0x7fd78c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd78c10ed40 0x7fd78c1a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 --2- 172.21.15.106:0/2074196541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd78c075390 0x7fd78c19ef00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:13.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 >> 172.21.15.106:0/2074196541 conn(0x7fd78c0fdcc0 msgr2=0x7fd78c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:13.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 shutdown_connections 2024-04-28T12:48:13.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:13.377+0000 7fd78be00640 1 -- 172.21.15.106:0/2074196541 wait complete. 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.171:6832/1724808557,v1:172.21.15.171:6833/1724808557] up:boot 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: fsmap 1 up:standby 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:13.460 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.171:6832/1724808557,v1:172.21.15.171:6833/1724808557] up:boot 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: fsmap 1 up:standby 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:14.432 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:14 smithi129 ceph-mon[62418]: Deploying daemon mds.k on smithi129 2024-04-28T12:48:14.432 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:14 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2074196541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:14 smithi171 ceph-mon[63127]: Deploying daemon mds.k on smithi129 2024-04-28T12:48:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:14 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2074196541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:14.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:14 smithi106 ceph-mon[28081]: Deploying daemon mds.k on smithi129 2024-04-28T12:48:14.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:14 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2074196541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:15.051 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 12:48:15 smithi129 podman[91922]: 2024-04-28 12:48:14.925784274 +0000 UTC m=+0.017958649 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:15.051 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 12:48:15 smithi129 podman[91922]: 2024-04-28 12:48:15.055757599 +0000 UTC m=+0.147931958 container create 3ebade331ac762b4877da2c2a267b85dc63ce670a99f9323ff77997ff00aa24d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T12:48:15.304 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:15 smithi129 ceph-mon[62418]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:15.396 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:15.476 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:15 smithi106 ceph-mon[28081]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:15 smithi171 ceph-mon[63127]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:15.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.620+0000 7fb0a0800640 1 -- 172.21.15.106:0/1144345494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb098105d80 msgr2=0x7fb098106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.620+0000 7fb0a0800640 1 --2- 172.21.15.106:0/1144345494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb098105d80 0x7fb098106200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb088009900 tx=0x7fb08802a410 comp rx=0 tx=0).stop 2024-04-28T12:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.620+0000 7fb0a0800640 1 -- 172.21.15.106:0/1144345494 shutdown_connections 2024-04-28T12:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.620+0000 7fb0a0800640 1 --2- 172.21.15.106:0/1144345494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb098105d80 0x7fb098106200 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.620+0000 7fb0a0800640 1 --2- 172.21.15.106:0/1144345494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb098106740 0x7fb09810cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.620+0000 7fb0a0800640 1 --2- 172.21.15.106:0/1144345494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb098104b90 0x7fb098104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.620+0000 7fb0a0800640 1 -- 172.21.15.106:0/1144345494 >> 172.21.15.106:0/1144345494 conn(0x7fb098100120 msgr2=0x7fb098102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.620+0000 7fb0a0800640 1 -- 172.21.15.106:0/1144345494 shutdown_connections 2024-04-28T12:48:15.617 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.620+0000 7fb0a0800640 1 -- 172.21.15.106:0/1144345494 wait complete. 2024-04-28T12:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.621+0000 7fb0a0800640 1 Processor -- start 2024-04-28T12:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.621+0000 7fb0a0800640 1 -- start start 2024-04-28T12:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb0a0800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb098104b90 0x7fb09819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb0a0800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb098105d80 0x7fb09819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb0a0800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb098106740 0x7fb0981a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:15.619 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb0a0800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb0981106a0 con 0x7fb098105d80 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb0a0800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb098110520 con 0x7fb098104b90 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb0a0800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb098110820 con 0x7fb098106740 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb098104b90 0x7fb09819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb098105d80 0x7fb09819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb098104b90 0x7fb09819ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:40830/0 (socket says 172.21.15.106:40830) 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09de00640 1 -- 172.21.15.106:0/4029403659 learned_addr learned my addr 172.21.15.106:0/4029403659 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09e800640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb098106740 0x7fb0981a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09de00640 1 -- 172.21.15.106:0/4029403659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb098106740 msgr2=0x7fb0981a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09de00640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb098106740 0x7fb0981a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09de00640 1 -- 172.21.15.106:0/4029403659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb098105d80 msgr2=0x7fb09819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09de00640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb098105d80 0x7fb09819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.622+0000 7fb09de00640 1 -- 172.21.15.106:0/4029403659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0981a2990 con 0x7fb098104b90 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.623+0000 7fb09de00640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb098104b90 0x7fb09819ad70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00e9c0 tx=0x7fb08c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.623+0000 7fb092a00640 1 -- 172.21.15.106:0/4029403659 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c00cd80 con 0x7fb098104b90 2024-04-28T12:48:15.620 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.623+0000 7fb092a00640 1 -- 172.21.15.106:0/4029403659 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb08c0035f0 con 0x7fb098104b90 2024-04-28T12:48:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.623+0000 7fb092a00640 1 -- 172.21.15.106:0/4029403659 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c010690 con 0x7fb098104b90 2024-04-28T12:48:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.623+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0981a2c80 con 0x7fb098104b90 2024-04-28T12:48:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.623+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0981a3150 con 0x7fb098104b90 2024-04-28T12:48:15.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.624+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb09810bf30 con 0x7fb098104b90 2024-04-28T12:48:15.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.627+0000 7fb092a00640 1 -- 172.21.15.106:0/4029403659 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb08c0037a0 con 0x7fb098104b90 2024-04-28T12:48:15.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.629+0000 7fb092a00640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb0800a3c30 0x7fb0800a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:15.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.630+0000 7fb092a00640 1 -- 172.21.15.106:0/4029403659 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb08c0d0300 con 0x7fb098104b90 2024-04-28T12:48:15.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.630+0000 7fb09d400640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb0800a3c30 0x7fb0800a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:15.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.630+0000 7fb09d400640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb0800a3c30 0x7fb0800a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb088000c00 tx=0x7fb0880314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:15.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.633+0000 7fb092a00640 1 -- 172.21.15.106:0/4029403659 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb08c09d0e0 con 0x7fb098104b90 2024-04-28T12:48:15.763 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 12:48:15 smithi129 podman[91922]: 2024-04-28 12:48:15.308979971 +0000 UTC m=+0.401154338 container init 3ebade331ac762b4877da2c2a267b85dc63ce670a99f9323ff77997ff00aa24d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T12:48:15.763 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 12:48:15 smithi129 podman[91922]: 2024-04-28 12:48:15.312493238 +0000 UTC m=+0.404667606 container start 3ebade331ac762b4877da2c2a267b85dc63ce670a99f9323ff77997ff00aa24d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-04-28T12:48:15.763 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 12:48:15 smithi129 ceph-mds[91942]: -- 172.21.15.129:0/3087787885 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561fc3535a40 con 0x561fc3508800 2024-04-28T12:48:15.764 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 12:48:15 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-k[91938]: starting mds.k at 2024-04-28T12:48:15.764 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 12:48:15 smithi129 bash[91922]: 3ebade331ac762b4877da2c2a267b85dc63ce670a99f9323ff77997ff00aa24d 2024-04-28T12:48:15.764 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 12:48:15 smithi129 systemd[1]: Started Ceph mds.k for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:15.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.881+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb09802dad0 con 0x7fb098104b90 2024-04-28T12:48:15.877 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.881+0000 7fb092a00640 1 -- 172.21.15.106:0/4029403659 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fb09802dad0 con 0x7fb098104b90 2024-04-28T12:48:15.877 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:15.878 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-28T12:48:15.878 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 2 2024-04-28T12:48:15.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb0800a3c30 msgr2=0x7fb0800a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb0800a3c30 0x7fb0800a60f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb088000c00 tx=0x7fb0880314a0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb098104b90 msgr2=0x7fb09819ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb098104b90 0x7fb09819ad70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00e9c0 tx=0x7fb08c00ee80 comp rx=0 tx=0).stop 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 shutdown_connections 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb0800a3c30 0x7fb0800a60f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb098105d80 0x7fb09819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb098106740 0x7fb0981a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 --2- 172.21.15.106:0/4029403659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb098104b90 0x7fb09819ad70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.884+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 >> 172.21.15.106:0/4029403659 conn(0x7fb098100120 msgr2=0x7fb098109100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.885+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 shutdown_connections 2024-04-28T12:48:15.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:15.885+0000 7fb0a0800640 1 -- 172.21.15.106:0/4029403659 wait complete. 2024-04-28T12:48:16.718 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:16.718 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:16.718 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:16.718 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:16.718 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:16.719 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:16.719 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: Deploying daemon mds.j on smithi106 2024-04-28T12:48:16.719 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/4029403659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:16.719 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:16.719 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.129:6832/2537709936,v1:172.21.15.129:6833/2537709936] up:boot 2024-04-28T12:48:16.719 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: fsmap 2 up:standby 2024-04-28T12:48:16.719 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:16 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-28T12:48:16.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:16.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:16.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:16.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:16.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:16.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:16.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: Deploying daemon mds.j on smithi106 2024-04-28T12:48:16.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/4029403659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:16.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:16.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.129:6832/2537709936,v1:172.21.15.129:6833/2537709936] up:boot 2024-04-28T12:48:16.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: fsmap 2 up:standby 2024-04-28T12:48:16.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:16 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-28T12:48:16.777 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:16.777 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:16.777 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:16.777 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:16.778 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:16.778 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:16.778 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: Deploying daemon mds.j on smithi106 2024-04-28T12:48:16.778 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4029403659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:16.778 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:16.778 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.129:6832/2537709936,v1:172.21.15.129:6833/2537709936] up:boot 2024-04-28T12:48:16.778 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: fsmap 2 up:standby 2024-04-28T12:48:16.778 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:16 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-28T12:48:17.899 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:18.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.108+0000 7f7539200640 1 -- 172.21.15.106:0/3596708672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7534105d20 msgr2=0x7f7534108110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:18.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.108+0000 7f7539200640 1 --2- 172.21.15.106:0/3596708672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7534105d20 0x7f7534108110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7520009900 tx=0x7f752002a410 comp rx=0 tx=0).stop 2024-04-28T12:48:18.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.108+0000 7f7539200640 1 -- 172.21.15.106:0/3596708672 shutdown_connections 2024-04-28T12:48:18.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.108+0000 7f7539200640 1 --2- 172.21.15.106:0/3596708672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7534105d20 0x7f7534108110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.108+0000 7f7539200640 1 --2- 172.21.15.106:0/3596708672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7534108650 0x7f753410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.108+0000 7f7539200640 1 --2- 172.21.15.106:0/3596708672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f75341033f0 0x7f75341057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.108+0000 7f7539200640 1 -- 172.21.15.106:0/3596708672 >> 172.21.15.106:0/3596708672 conn(0x7f75340fcde0 msgr2=0x7f75340ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:18.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.109+0000 7f7539200640 1 -- 172.21.15.106:0/3596708672 shutdown_connections 2024-04-28T12:48:18.105 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.109+0000 7f7539200640 1 -- 172.21.15.106:0/3596708672 wait complete. 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.109+0000 7f7539200640 1 Processor -- start 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7539200640 1 -- start start 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7539200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75341033f0 0x7f753419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7539200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7534105d20 0x7f753419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7539200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7534108650 0x7f75341a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7539200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f753410e4a0 con 0x7f75341033f0 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7539200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f753410e320 con 0x7f7534108650 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7539200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f753410e620 con 0x7f7534105d20 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7531e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75341033f0 0x7f753419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7531e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75341033f0 0x7f753419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47700/0 (socket says 172.21.15.106:47700) 2024-04-28T12:48:18.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7531e00640 1 -- 172.21.15.106:0/4087493551 learned_addr learned my addr 172.21.15.106:0/4087493551 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.110+0000 7f7532800640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7534108650 0x7f75341a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f7531400640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7534105d20 0x7f753419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f7531e00640 1 -- 172.21.15.106:0/4087493551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7534105d20 msgr2=0x7f753419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f7531e00640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7534105d20 0x7f753419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f7531e00640 1 -- 172.21.15.106:0/4087493551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7534108650 msgr2=0x7f75341a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f7531e00640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7534108650 0x7f75341a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f7531e00640 1 -- 172.21.15.106:0/4087493551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75341a2a30 con 0x7f75341033f0 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f7532800640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7534108650 0x7f75341a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f7531e00640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75341033f0 0x7f753419ae10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f752400ea60 tx=0x7f752400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f752aa00640 1 -- 172.21.15.106:0/4087493551 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400cde0 con 0x7f75341033f0 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f752aa00640 1 -- 172.21.15.106:0/4087493551 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75240035f0 con 0x7f75341033f0 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.111+0000 7f752aa00640 1 -- 172.21.15.106:0/4087493551 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524010690 con 0x7f75341033f0 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.112+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75341a2d20 con 0x7f75341033f0 2024-04-28T12:48:18.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.112+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75341a3190 con 0x7f75341033f0 2024-04-28T12:48:18.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.113+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753410a1a0 con 0x7f75341033f0 2024-04-28T12:48:18.110 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.115+0000 7f752aa00640 1 -- 172.21.15.106:0/4087493551 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f75240037a0 con 0x7f75341033f0 2024-04-28T12:48:18.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.117+0000 7f752aa00640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75180a3c30 0x7f75180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:18.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.117+0000 7f752aa00640 1 -- 172.21.15.106:0/4087493551 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f75240d0240 con 0x7f75341033f0 2024-04-28T12:48:18.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.117+0000 7f7531400640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75180a3c30 0x7f75180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:18.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.118+0000 7f7531400640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75180a3c30 0x7f75180a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7520009230 tx=0x7f75200314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:18.116 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.121+0000 7f752aa00640 1 -- 172.21.15.106:0/4087493551 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f752409d020 con 0x7f75341033f0 2024-04-28T12:48:18.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.367+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75340635d0 con 0x7f75341033f0 2024-04-28T12:48:18.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.368+0000 7f752aa00640 1 -- 172.21.15.106:0/4087493551 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f752409c920 con 0x7f75341033f0 2024-04-28T12:48:18.364 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:18.364 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-28T12:48:18.364 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 3 2024-04-28T12:48:18.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.370+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75180a3c30 msgr2=0x7f75180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:18.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.370+0000 7f7539200640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75180a3c30 0x7f75180a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7520009230 tx=0x7f75200314a0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.370+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75341033f0 msgr2=0x7f753419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.370+0000 7f7539200640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75341033f0 0x7f753419ae10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f752400ea60 tx=0x7f752400ef20 comp rx=0 tx=0).stop 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.370+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 shutdown_connections 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.370+0000 7f7539200640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75180a3c30 0x7f75180a60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.370+0000 7f7539200640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7534108650 0x7f75341a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.370+0000 7f7539200640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7534105d20 0x7f753419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.370+0000 7f7539200640 1 --2- 172.21.15.106:0/4087493551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75341033f0 0x7f753419ae10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.371+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 >> 172.21.15.106:0/4087493551 conn(0x7f75340fcde0 msgr2=0x7f7534102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.371+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 shutdown_connections 2024-04-28T12:48:18.367 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:18.371+0000 7f7539200640 1 -- 172.21.15.106:0/4087493551 wait complete. 2024-04-28T12:48:18.947 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 12:48:18 smithi106 podman[67913]: 2024-04-28 12:48:18.680661574 +0000 UTC m=+0.151499781 container create 4f9290cb23b564e8a70e40ab24ba0c52d39aecaf3c62e2babdf9f1b73f883df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-28T12:48:18.948 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 12:48:18 smithi106 podman[67913]: 2024-04-28 12:48:18.951579192 +0000 UTC m=+0.422417433 container init 4f9290cb23b564e8a70e40ab24ba0c52d39aecaf3c62e2babdf9f1b73f883df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T12:48:19.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:19 smithi106 ceph-mon[28081]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:19.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:19 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4087493551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:19.326 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 12:48:18 smithi106 podman[67913]: 2024-04-28 12:48:18.961328045 +0000 UTC m=+0.432166286 container start 4f9290cb23b564e8a70e40ab24ba0c52d39aecaf3c62e2babdf9f1b73f883df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-28T12:48:19.326 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 12:48:19 smithi106 ceph-mds[67932]: -- 172.21.15.106:0/2459695757 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f4a901a40 con 0x564f4a8d4c00 2024-04-28T12:48:19.326 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 12:48:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j[67928]: starting mds.j at 2024-04-28T12:48:19.326 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 12:48:19 smithi106 bash[67913]: 4f9290cb23b564e8a70e40ab24ba0c52d39aecaf3c62e2babdf9f1b73f883df9 2024-04-28T12:48:19.326 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 12:48:19 smithi106 systemd[1]: Started Ceph mds.j for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:19 smithi171 ceph-mon[63127]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:19 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/4087493551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:19.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:19 smithi129 ceph-mon[62418]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:19.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:19 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/4087493551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:20.387 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:20.437 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:20 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6834/3035871722,v1:172.21.15.106:6835/3035871722] up:boot 2024-04-28T12:48:20.437 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:20 smithi106 ceph-mon[28081]: fsmap 3 up:standby 2024-04-28T12:48:20.437 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:20 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-28T12:48:20.437 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:20 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:20.437 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:20 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:20.437 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:20 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:20.437 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:20 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:20.437 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:20 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:20.438 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:20 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:20.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:20 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6834/3035871722,v1:172.21.15.106:6835/3035871722] up:boot 2024-04-28T12:48:20.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:20 smithi171 ceph-mon[63127]: fsmap 3 up:standby 2024-04-28T12:48:20.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:20 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-28T12:48:20.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:20 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:20.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:20 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:20.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:20 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:20.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:20 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:20.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:20 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:20.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:20 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:20.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:20 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6834/3035871722,v1:172.21.15.106:6835/3035871722] up:boot 2024-04-28T12:48:20.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:20 smithi129 ceph-mon[62418]: fsmap 3 up:standby 2024-04-28T12:48:20.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:20 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-28T12:48:20.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:20 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:20.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:20 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:20.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:20 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:20.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:20 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:20.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:20 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:20.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:20 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:20.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.607+0000 7f184f200640 1 -- 172.21.15.106:0/4048792218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 msgr2=0x7f1850102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:20.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.607+0000 7f184f200640 1 --2- 172.21.15.106:0/4048792218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 0x7f1850102b90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1840009860 tx=0x7f184002a340 comp rx=0 tx=0).stop 2024-04-28T12:48:20.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.608+0000 7f184f200640 1 -- 172.21.15.106:0/4048792218 shutdown_connections 2024-04-28T12:48:20.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.608+0000 7f184f200640 1 --2- 172.21.15.106:0/4048792218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850104320 0x7f185010abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.608+0000 7f184f200640 1 --2- 172.21.15.106:0/4048792218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1850103960 0x7f1850103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.608+0000 7f184f200640 1 --2- 172.21.15.106:0/4048792218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 0x7f1850102b90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.608+0000 7f184f200640 1 -- 172.21.15.106:0/4048792218 >> 172.21.15.106:0/4048792218 conn(0x7f18500fdcc0 msgr2=0x7f1850100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:20.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.608+0000 7f184f200640 1 -- 172.21.15.106:0/4048792218 shutdown_connections 2024-04-28T12:48:20.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.608+0000 7f184f200640 1 -- 172.21.15.106:0/4048792218 wait complete. 2024-04-28T12:48:20.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.609+0000 7f184f200640 1 Processor -- start 2024-04-28T12:48:20.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.609+0000 7f184f200640 1 -- start start 2024-04-28T12:48:20.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 0x7f1850198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:20.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1850103960 0x7f1850198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:20.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850104320 0x7f185019fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f185010e270 con 0x7f1850104320 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f185010e0f0 con 0x7f1850102770 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f185010e3f0 con 0x7f1850103960 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 0x7f1850198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184d400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1850103960 0x7f1850198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 0x7f1850198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:51400/0 (socket says 172.21.15.106:51400) 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184de00640 1 -- 172.21.15.106:0/2396199286 learned_addr learned my addr 172.21.15.106:0/2396199286 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184e800640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850104320 0x7f185019fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184de00640 1 -- 172.21.15.106:0/2396199286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1850103960 msgr2=0x7f1850198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184de00640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1850103960 0x7f1850198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184de00640 1 -- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850104320 msgr2=0x7f185019fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184de00640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850104320 0x7f185019fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184de00640 1 -- 172.21.15.106:0/2396199286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18501a0550 con 0x7f1850102770 2024-04-28T12:48:20.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.610+0000 7f184e800640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850104320 0x7f185019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:48:20.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.611+0000 7f184de00640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 0x7f1850198930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18400057c0 tx=0x7f1840032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:20.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.611+0000 7f1846a00640 1 -- 172.21.15.106:0/2396199286 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840037040 con 0x7f1850102770 2024-04-28T12:48:20.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.611+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18501a07e0 con 0x7f1850102770 2024-04-28T12:48:20.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.611+0000 7f1846a00640 1 -- 172.21.15.106:0/2396199286 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1840030bd0 con 0x7f1850102770 2024-04-28T12:48:20.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.611+0000 7f1846a00640 1 -- 172.21.15.106:0/2396199286 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840005280 con 0x7f1850102770 2024-04-28T12:48:20.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.611+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501a0c70 con 0x7f1850102770 2024-04-28T12:48:20.608 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.612+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1850109a90 con 0x7f1850102770 2024-04-28T12:48:20.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.614+0000 7f1846a00640 1 -- 172.21.15.106:0/2396199286 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1840043020 con 0x7f1850102770 2024-04-28T12:48:20.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.617+0000 7f1846a00640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18340a3bb0 0x7f18340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:20.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.617+0000 7f1846a00640 1 -- 172.21.15.106:0/2396199286 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f18400ef870 con 0x7f1850102770 2024-04-28T12:48:20.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.617+0000 7f184d400640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18340a3bb0 0x7f18340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:20.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.618+0000 7f184d400640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18340a3bb0 0x7f18340a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1850199e40 tx=0x7f183c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:20.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.620+0000 7f1846a00640 1 -- 172.21.15.106:0/2396199286 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f18400bc650 con 0x7f1850102770 2024-04-28T12:48:20.861 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.866+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f185002dad0 con 0x7f1850102770 2024-04-28T12:48:20.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.867+0000 7f1846a00640 1 -- 172.21.15.106:0/2396199286 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3052 (secure 0 0 0) 0x7f18400bbf50 con 0x7f1850102770 2024-04-28T12:48:20.863 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:20.863 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[]} 2024-04-28T12:48:20.863 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 4 2024-04-28T12:48:20.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18340a3bb0 msgr2=0x7f18340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:20.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18340a3bb0 0x7f18340a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1850199e40 tx=0x7f183c005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:20.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 msgr2=0x7f1850198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:20.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 0x7f1850198930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f18400057c0 tx=0x7f1840032da0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 shutdown_connections 2024-04-28T12:48:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18340a3bb0 0x7f18340a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1850104320 0x7f185019fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1850103960 0x7f1850198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 --2- 172.21.15.106:0/2396199286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1850102770 0x7f1850198930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.869+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 >> 172.21.15.106:0/2396199286 conn(0x7f18500fdcc0 msgr2=0x7f1850106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.870+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 shutdown_connections 2024-04-28T12:48:20.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:20.870+0000 7f184f200640 1 -- 172.21.15.106:0/2396199286 wait complete. 2024-04-28T12:48:21.220 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:21 smithi106 ceph-mon[28081]: Deploying daemon mds.d on smithi106 2024-04-28T12:48:21.220 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:21 smithi106 ceph-mon[28081]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:21.220 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:21 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2396199286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:21 smithi171 ceph-mon[63127]: Deploying daemon mds.d on smithi106 2024-04-28T12:48:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:21 smithi171 ceph-mon[63127]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:21 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2396199286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:21.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:21 smithi129 ceph-mon[62418]: Deploying daemon mds.d on smithi106 2024-04-28T12:48:21.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:21 smithi129 ceph-mon[62418]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:21.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:21 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2396199286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:22.759 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 12:48:22 smithi106 systemd[1]: Starting Ceph mds.d for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:48:22.886 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:23.037 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 12:48:22 smithi106 podman[68253]: 2024-04-28 12:48:22.667066258 +0000 UTC m=+0.038168809 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:23.037 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 12:48:22 smithi106 podman[68253]: 2024-04-28 12:48:22.787200088 +0000 UTC m=+0.158302690 container create 058a2071450b79ce392eea528d6cb2b221358fb5f2e11686cd9d4ecbb76b8d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-28T12:48:23.037 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 12:48:23 smithi106 podman[68253]: 2024-04-28 12:48:23.040674084 +0000 UTC m=+0.411776683 container init 058a2071450b79ce392eea528d6cb2b221358fb5f2e11686cd9d4ecbb76b8d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-28T12:48:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.092+0000 7fdb3e400640 1 -- 172.21.15.106:0/3736427742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb38108630 msgr2=0x7fdb3810aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.092+0000 7fdb3e400640 1 --2- 172.21.15.106:0/3736427742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb38108630 0x7fdb3810aa80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009ca0 tx=0x7fdb3002a8c0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.092+0000 7fdb3e400640 1 -- 172.21.15.106:0/3736427742 shutdown_connections 2024-04-28T12:48:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.092+0000 7fdb3e400640 1 --2- 172.21.15.106:0/3736427742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb38108630 0x7fdb3810aa80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.092+0000 7fdb3e400640 1 --2- 172.21.15.106:0/3736427742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb38105d00 0x7fdb381080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.092+0000 7fdb3e400640 1 --2- 172.21.15.106:0/3736427742 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb381033d0 0x7fdb381057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.092+0000 7fdb3e400640 1 -- 172.21.15.106:0/3736427742 >> 172.21.15.106:0/3736427742 conn(0x7fdb380fcdc0 msgr2=0x7fdb380ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.092+0000 7fdb3e400640 1 -- 172.21.15.106:0/3736427742 shutdown_connections 2024-04-28T12:48:23.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.092+0000 7fdb3e400640 1 -- 172.21.15.106:0/3736427742 wait complete. 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.093+0000 7fdb3e400640 1 Processor -- start 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.093+0000 7fdb3e400640 1 -- start start 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb3e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb381033d0 0x7fdb3819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb3e400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb38105d00 0x7fdb3819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb3e400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb38108630 0x7fdb381a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb3e400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810e4f0 con 0x7fdb381033d0 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb3e400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810e370 con 0x7fdb38108630 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb3e400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3810e670 con 0x7fdb38105d00 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb37400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb381033d0 0x7fdb3819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb37400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb381033d0 0x7fdb3819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33756/0 (socket says 172.21.15.106:33756) 2024-04-28T12:48:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb37400640 1 -- 172.21.15.106:0/1209319402 learned_addr learned my addr 172.21.15.106:0/1209319402 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb37e00640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb38108630 0x7fdb381a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.094+0000 7fdb36a00640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb38105d00 0x7fdb3819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb37e00640 1 -- 172.21.15.106:0/1209319402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb38105d00 msgr2=0x7fdb3819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb37e00640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb38105d00 0x7fdb3819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb37e00640 1 -- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb381033d0 msgr2=0x7fdb3819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb37e00640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb381033d0 0x7fdb3819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb37e00640 1 -- 172.21.15.106:0/1209319402 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb381a29b0 con 0x7fdb38108630 2024-04-28T12:48:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb37400640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb381033d0 0x7fdb3819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb37e00640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb38108630 0x7fdb381a2310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdb30031a20 tx=0x7fdb300316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb2fe00640 1 -- 172.21.15.106:0/1209319402 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30003b70 con 0x7fdb38108630 2024-04-28T12:48:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb2fe00640 1 -- 172.21.15.106:0/1209319402 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb30003d10 con 0x7fdb38108630 2024-04-28T12:48:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.095+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb381a2c40 con 0x7fdb38108630 2024-04-28T12:48:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.096+0000 7fdb2fe00640 1 -- 172.21.15.106:0/1209319402 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3002fae0 con 0x7fdb38108630 2024-04-28T12:48:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.096+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb381a3190 con 0x7fdb38108630 2024-04-28T12:48:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.097+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3810a120 con 0x7fdb38108630 2024-04-28T12:48:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.099+0000 7fdb2fe00640 1 -- 172.21.15.106:0/1209319402 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdb30004610 con 0x7fdb38108630 2024-04-28T12:48:23.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.100+0000 7fdb2fe00640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdb1c0a3bb0 0x7fdb1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:23.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.100+0000 7fdb2fe00640 1 -- 172.21.15.106:0/1209319402 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fdb300f13d0 con 0x7fdb38108630 2024-04-28T12:48:23.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.101+0000 7fdb37400640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdb1c0a3bb0 0x7fdb1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:23.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.101+0000 7fdb37400640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdb1c0a3bb0 0x7fdb1c0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb3819c270 tx=0x7fdb28005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:23.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.105+0000 7fdb2fe00640 1 -- 172.21.15.106:0/1209319402 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdb300be1b0 con 0x7fdb38108630 2024-04-28T12:48:23.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:23 smithi106 ceph-mon[28081]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:23.324 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 12:48:23 smithi106 podman[68253]: 2024-04-28 12:48:23.052330528 +0000 UTC m=+0.423433113 container start 058a2071450b79ce392eea528d6cb2b221358fb5f2e11686cd9d4ecbb76b8d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-28T12:48:23.325 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 12:48:23 smithi106 ceph-mds[68292]: -- 172.21.15.106:0/3058096151 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55699454fa40 con 0x556994522800 2024-04-28T12:48:23.325 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 12:48:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d[68273]: starting mds.d at 2024-04-28T12:48:23.325 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 12:48:23 smithi106 bash[68253]: 058a2071450b79ce392eea528d6cb2b221358fb5f2e11686cd9d4ecbb76b8d29 2024-04-28T12:48:23.325 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 12:48:23 smithi106 systemd[1]: Started Ceph mds.d for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:23.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.354+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb38063530 con 0x7fdb38108630 2024-04-28T12:48:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.355+0000 7fdb2fe00640 1 -- 172.21.15.106:0/1209319402 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3880 (secure 0 0 0) 0x7fdb300bdab0 con 0x7fdb38108630 2024-04-28T12:48:23.351 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:23.351 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-28T12:48:23.351 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 5 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.357+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdb1c0a3bb0 msgr2=0x7fdb1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.357+0000 7fdb3e400640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdb1c0a3bb0 0x7fdb1c0a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdb3819c270 tx=0x7fdb28005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb38108630 msgr2=0x7fdb381a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb38108630 0x7fdb381a2310 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdb30031a20 tx=0x7fdb300316f0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 shutdown_connections 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdb1c0a3bb0 0x7fdb1c0a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb38108630 0x7fdb381a2310 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb38105d00 0x7fdb3819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 --2- 172.21.15.106:0/1209319402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdb381033d0 0x7fdb3819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:23.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 >> 172.21.15.106:0/1209319402 conn(0x7fdb380fcdc0 msgr2=0x7fdb38102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:23.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 shutdown_connections 2024-04-28T12:48:23.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:23.358+0000 7fdb3e400640 1 -- 172.21.15.106:0/1209319402 wait complete. 2024-04-28T12:48:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:23 smithi171 ceph-mon[63127]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:23.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:23 smithi129 ceph-mon[62418]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:24.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6836/4131098763,v1:172.21.15.106:6837/4131098763] up:boot 2024-04-28T12:48:24.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: fsmap 4 up:standby 2024-04-28T12:48:24.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-28T12:48:24.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:24.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:24.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:24.216 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:24.217 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:24.217 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:24.217 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:24 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1209319402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:24.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6836/4131098763,v1:172.21.15.106:6837/4131098763] up:boot 2024-04-28T12:48:24.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: fsmap 4 up:standby 2024-04-28T12:48:24.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-28T12:48:24.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:24.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:24.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:24.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:24.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:24.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:24.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:24 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1209319402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:24.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6836/4131098763,v1:172.21.15.106:6837/4131098763] up:boot 2024-04-28T12:48:24.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: fsmap 4 up:standby 2024-04-28T12:48:24.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-28T12:48:24.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:24.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:24.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:24.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:24.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:24.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:24.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:24 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1209319402' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:25.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:25 smithi129 ceph-mon[62418]: Deploying daemon mds.e on smithi129 2024-04-28T12:48:25.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:25 smithi129 ceph-mon[62418]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:25.374 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:25.452 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:25 smithi106 ceph-mon[28081]: Deploying daemon mds.e on smithi129 2024-04-28T12:48:25.452 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:25 smithi106 ceph-mon[28081]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:25 smithi171 ceph-mon[63127]: Deploying daemon mds.e on smithi129 2024-04-28T12:48:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:25 smithi171 ceph-mon[63127]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:25.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.593+0000 7f99d0000640 1 -- 172.21.15.106:0/2475351396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8102970 msgr2=0x7f99c8102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:25.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.593+0000 7f99d0000640 1 --2- 172.21.15.106:0/2475351396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8102970 0x7f99c8102d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009900 tx=0x7f99bc02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.593+0000 7f99d0000640 1 -- 172.21.15.106:0/2475351396 shutdown_connections 2024-04-28T12:48:25.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.593+0000 7f99d0000640 1 --2- 172.21.15.106:0/2475351396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c8103b60 0x7f99c8103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.593+0000 7f99d0000640 1 --2- 172.21.15.106:0/2475351396 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c8104520 0x7f99c810adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.593+0000 7f99d0000640 1 --2- 172.21.15.106:0/2475351396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8102970 0x7f99c8102d90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.593+0000 7f99d0000640 1 -- 172.21.15.106:0/2475351396 >> 172.21.15.106:0/2475351396 conn(0x7f99c80fdee0 msgr2=0x7f99c8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:25.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.593+0000 7f99d0000640 1 -- 172.21.15.106:0/2475351396 shutdown_connections 2024-04-28T12:48:25.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.593+0000 7f99d0000640 1 -- 172.21.15.106:0/2475351396 wait complete. 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.594+0000 7f99d0000640 1 Processor -- start 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.594+0000 7f99d0000640 1 -- start start 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.594+0000 7f99d0000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c8102970 0x7f99c8198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.594+0000 7f99d0000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8103b60 0x7f99c8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.594+0000 7f99d0000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c8104520 0x7f99c81a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.594+0000 7f99d0000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99c810e4c0 con 0x7f99c8102970 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.594+0000 7f99d0000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99c810e340 con 0x7f99c8103b60 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.594+0000 7f99d0000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99c810e640 con 0x7f99c8104520 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.594+0000 7f99cd600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c8102970 0x7f99c8198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ccc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8103b60 0x7f99c8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ccc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8103b60 0x7f99c8199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:51456/0 (socket says 172.21.15.106:51456) 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ccc00640 1 -- 172.21.15.106:0/304740494 learned_addr learned my addr 172.21.15.106:0/304740494 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ce000640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c8104520 0x7f99c81a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:25.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ccc00640 1 -- 172.21.15.106:0/304740494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c8104520 msgr2=0x7f99c81a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ccc00640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c8104520 0x7f99c81a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ccc00640 1 -- 172.21.15.106:0/304740494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c8102970 msgr2=0x7f99c8198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ccc00640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c8102970 0x7f99c8198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ccc00640 1 -- 172.21.15.106:0/304740494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c81a07e0 con 0x7f99c8103b60 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ce000640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c8104520 0x7f99c81a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99ccc00640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8103b60 0x7f99c8199100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99b400e9f0 tx=0x7f99b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99c6000640 1 -- 172.21.15.106:0/304740494 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b400cd80 con 0x7f99c8103b60 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99c6000640 1 -- 172.21.15.106:0/304740494 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99b40108e0 con 0x7f99c8103b60 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99c6000640 1 -- 172.21.15.106:0/304740494 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b4018980 con 0x7f99c8103b60 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.595+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c81a0ad0 con 0x7f99c8103b60 2024-04-28T12:48:25.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.596+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c81a0fd0 con 0x7f99c8103b60 2024-04-28T12:48:25.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.597+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c8109d10 con 0x7f99c8103b60 2024-04-28T12:48:25.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.599+0000 7f99c6000640 1 -- 172.21.15.106:0/304740494 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f99b4010430 con 0x7f99c8103b60 2024-04-28T12:48:25.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.600+0000 7f99c6000640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99b00a3c30 0x7f99b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:25.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.600+0000 7f99c6000640 1 -- 172.21.15.106:0/304740494 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f99b40d07f0 con 0x7f99c8103b60 2024-04-28T12:48:25.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.600+0000 7f99cd600640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99b00a3c30 0x7f99b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:25.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.601+0000 7f99cd600640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99b00a3c30 0x7f99b00a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005ab0 tx=0x7f99bc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:25.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.602+0000 7f99c6000640 1 -- 172.21.15.106:0/304740494 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f99b409d5d0 con 0x7f99c8103b60 2024-04-28T12:48:25.696 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 12:48:25 smithi129 systemd[1]: Starting Ceph mds.e for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:48:25.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.851+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99c802dd20 con 0x7f99c8103b60 2024-04-28T12:48:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.852+0000 7f99c6000640 1 -- 172.21.15.106:0/304740494 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3880 (secure 0 0 0) 0x7f99b409ced0 con 0x7f99c8103b60 2024-04-28T12:48:25.848 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-28T12:48:25.848 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 5 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99b00a3c30 msgr2=0x7f99b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99b00a3c30 0x7f99b00a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f99bc005ab0 tx=0x7f99bc005a20 comp rx=0 tx=0).stop 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8103b60 msgr2=0x7f99c8199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8103b60 0x7f99c8199100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99b400e9f0 tx=0x7f99b400eeb0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 shutdown_connections 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99b00a3c30 0x7f99b00a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99c8104520 0x7f99c81a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99c8103b60 0x7f99c8199100 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 --2- 172.21.15.106:0/304740494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99c8102970 0x7f99c8198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:25.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 >> 172.21.15.106:0/304740494 conn(0x7f99c80fdee0 msgr2=0x7f99c8106ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 shutdown_connections 2024-04-28T12:48:25.852 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:25.855+0000 7f99d0000640 1 -- 172.21.15.106:0/304740494 wait complete. 2024-04-28T12:48:25.983 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 12:48:25 smithi129 podman[92242]: 2024-04-28 12:48:25.603700886 +0000 UTC m=+0.018716982 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:25.984 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 12:48:25 smithi129 podman[92242]: 2024-04-28 12:48:25.752618502 +0000 UTC m=+0.167634578 container create 958082fb42cbf244b7490cb3166583c7ea6c035f30c2decb4cc7910302c9e275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T12:48:25.984 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 12:48:25 smithi129 podman[92242]: 2024-04-28 12:48:25.988587642 +0000 UTC m=+0.403603719 container init 958082fb42cbf244b7490cb3166583c7ea6c035f30c2decb4cc7910302c9e275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-28T12:48:26.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:26 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/304740494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:26.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:26 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:26.263 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 12:48:25 smithi129 podman[92242]: 2024-04-28 12:48:25.991172335 +0000 UTC m=+0.406188420 container start 958082fb42cbf244b7490cb3166583c7ea6c035f30c2decb4cc7910302c9e275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-28T12:48:26.264 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 12:48:26 smithi129 ceph-mds[92260]: -- 172.21.15.129:0/1927836703 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556fbe20da40 con 0x556fbe1e1000 2024-04-28T12:48:26.264 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 12:48:26 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e[92256]: starting mds.e at 2024-04-28T12:48:26.264 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 12:48:26 smithi129 bash[92242]: 958082fb42cbf244b7490cb3166583c7ea6c035f30c2decb4cc7910302c9e275 2024-04-28T12:48:26.264 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 12:48:26 smithi129 systemd[1]: Started Ceph mds.e for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:26 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/304740494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:26 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:26.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:26 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/304740494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:26.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:26 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:27.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:27 smithi171 ceph-mon[63127]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:27.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:27.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:27 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.129:6834/1242652051,v1:172.21.15.129:6835/1242652051] up:boot 2024-04-28T12:48:27.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:27 smithi171 ceph-mon[63127]: fsmap 5 up:standby 2024-04-28T12:48:27.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-28T12:48:27.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:27.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:27.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:27.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:27 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:27 smithi129 ceph-mon[62418]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:27 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.129:6834/1242652051,v1:172.21.15.129:6835/1242652051] up:boot 2024-04-28T12:48:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:27 smithi129 ceph-mon[62418]: fsmap 5 up:standby 2024-04-28T12:48:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-28T12:48:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:27 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:27 smithi106 ceph-mon[28081]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:27 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.129:6834/1242652051,v1:172.21.15.129:6835/1242652051] up:boot 2024-04-28T12:48:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:27 smithi106 ceph-mon[28081]: fsmap 5 up:standby 2024-04-28T12:48:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-28T12:48:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:27 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:27.871 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:28.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.085+0000 7ff6ec200640 1 -- 172.21.15.106:0/1830540908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410a180 msgr2=0x7ff6e410a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:28.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.085+0000 7ff6ec200640 1 --2- 172.21.15.106:0/1830540908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410a180 0x7ff6e410a600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009ca0 tx=0x7ff6d802a8c0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.085+0000 7ff6ec200640 1 -- 172.21.15.106:0/1830540908 shutdown_connections 2024-04-28T12:48:28.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.085+0000 7ff6ec200640 1 --2- 172.21.15.106:0/1830540908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410a180 0x7ff6e410a600 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.085+0000 7ff6ec200640 1 --2- 172.21.15.106:0/1830540908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6e410ab40 0x7ff6e41113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.085+0000 7ff6ec200640 1 --2- 172.21.15.106:0/1830540908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e4108f90 0x7ff6e41093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.085+0000 7ff6ec200640 1 -- 172.21.15.106:0/1830540908 >> 172.21.15.106:0/1830540908 conn(0x7ff6e4077f40 msgr2=0x7ff6e407a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:28.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.085+0000 7ff6ec200640 1 -- 172.21.15.106:0/1830540908 shutdown_connections 2024-04-28T12:48:28.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.085+0000 7ff6ec200640 1 -- 172.21.15.106:0/1830540908 wait complete. 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.086+0000 7ff6ec200640 1 Processor -- start 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ec200640 1 -- start start 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ec200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e4108f90 0x7ff6e419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ec200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6e410a180 0x7ff6e419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ec200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410ab40 0x7ff6e41a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ec200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4114b20 con 0x7ff6e410ab40 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ec200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e41149a0 con 0x7ff6e410a180 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ec200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e4114ca0 con 0x7ff6e4108f90 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ea200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410ab40 0x7ff6e41a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ea200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410ab40 0x7ff6e41a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33800/0 (socket says 172.21.15.106:33800) 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6ea200640 1 -- 172.21.15.106:0/100710004 learned_addr learned my addr 172.21.15.106:0/100710004 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:28.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6e9800640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e4108f90 0x7ff6e419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.087+0000 7ff6e8e00640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6e410a180 0x7ff6e419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6ea200640 1 -- 172.21.15.106:0/100710004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e4108f90 msgr2=0x7ff6e419f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6ea200640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e4108f90 0x7ff6e419f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6ea200640 1 -- 172.21.15.106:0/100710004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6e410a180 msgr2=0x7ff6e419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6ea200640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6e410a180 0x7ff6e419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6ea200640 1 -- 172.21.15.106:0/100710004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e41a6dd0 con 0x7ff6e410ab40 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6ea200640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410ab40 0x7ff6e41a6760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00e9b0 tx=0x7ff6dc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6e2000640 1 -- 172.21.15.106:0/100710004 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc00cd80 con 0x7ff6e410ab40 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6e2000640 1 -- 172.21.15.106:0/100710004 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6dc0035f0 con 0x7ff6e410ab40 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e41a70c0 con 0x7ff6e410ab40 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.088+0000 7ff6e2000640 1 -- 172.21.15.106:0/100710004 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc010690 con 0x7ff6e410ab40 2024-04-28T12:48:28.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.089+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e41a75e0 con 0x7ff6e410ab40 2024-04-28T12:48:28.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.090+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e41102b0 con 0x7ff6e410ab40 2024-04-28T12:48:28.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.092+0000 7ff6e2000640 1 -- 172.21.15.106:0/100710004 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff6dc004020 con 0x7ff6e410ab40 2024-04-28T12:48:28.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.093+0000 7ff6e2000640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff6cc0a3c30 0x7ff6cc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:28.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.093+0000 7ff6e2000640 1 -- 172.21.15.106:0/100710004 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff6dc0d0270 con 0x7ff6e410ab40 2024-04-28T12:48:28.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.093+0000 7ff6e9800640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff6cc0a3c30 0x7ff6cc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:28.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.094+0000 7ff6e9800640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff6cc0a3c30 0x7ff6cc0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4109fe0 tx=0x7ff6d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:28.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.097+0000 7ff6e2000640 1 -- 172.21.15.106:0/100710004 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff6dc09d050 con 0x7ff6e410ab40 2024-04-28T12:48:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:28 smithi106 ceph-mon[28081]: Deploying daemon mds.f on smithi171 2024-04-28T12:48:28.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.340+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6e402dad0 con 0x7ff6e410ab40 2024-04-28T12:48:28.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.341+0000 7ff6e2000640 1 -- 172.21.15.106:0/100710004 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4708 (secure 0 0 0) 0x7ff6dc09c950 con 0x7ff6e410ab40 2024-04-28T12:48:28.337 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:28.338 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-28T12:48:28.338 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 6 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff6cc0a3c30 msgr2=0x7ff6cc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff6cc0a3c30 0x7ff6cc0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4109fe0 tx=0x7ff6d4005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410ab40 msgr2=0x7ff6e41a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410ab40 0x7ff6e41a6760 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00e9b0 tx=0x7ff6dc00ee70 comp rx=0 tx=0).stop 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 shutdown_connections 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff6cc0a3c30 0x7ff6cc0a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6e410ab40 0x7ff6e41a6760 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6e410a180 0x7ff6e419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 --2- 172.21.15.106:0/100710004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e4108f90 0x7ff6e419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 >> 172.21.15.106:0/100710004 conn(0x7ff6e4077f40 msgr2=0x7ff6e410d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:28.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 shutdown_connections 2024-04-28T12:48:28.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:28.344+0000 7ff6ec200640 1 -- 172.21.15.106:0/100710004 wait complete. 2024-04-28T12:48:28.479 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:28 smithi171 ceph-mon[63127]: Deploying daemon mds.f on smithi171 2024-04-28T12:48:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:28 smithi129 ceph-mon[62418]: Deploying daemon mds.f on smithi171 2024-04-28T12:48:28.735 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 12:48:28 smithi171 podman[92774]: 2024-04-28 12:48:28.612986754 +0000 UTC m=+0.026132376 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:28.735 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 12:48:28 smithi171 podman[92774]: 2024-04-28 12:48:28.740052703 +0000 UTC m=+0.153198314 container create 82ae243700063fec78a8798001178ddcf1cf26fc1a77ed92aa8e5265d52f8ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-28T12:48:28.995 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 12:48:28 smithi171 podman[92774]: 2024-04-28 12:48:28.976322073 +0000 UTC m=+0.389467681 container init 82ae243700063fec78a8798001178ddcf1cf26fc1a77ed92aa8e5265d52f8ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T12:48:28.995 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 12:48:28 smithi171 podman[92774]: 2024-04-28 12:48:28.978961149 +0000 UTC m=+0.392106757 container start 82ae243700063fec78a8798001178ddcf1cf26fc1a77ed92aa8e5265d52f8ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T12:48:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:29 smithi171 ceph-mon[63127]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:29 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/100710004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:29 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:29 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:29.495 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 12:48:29 smithi171 ceph-mds[92793]: -- 172.21.15.171:0/4284099896 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ee2417a40 con 0x563ee23eb000 2024-04-28T12:48:29.496 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 12:48:29 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f[92789]: starting mds.f at 2024-04-28T12:48:29.496 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 12:48:29 smithi171 bash[92774]: 82ae243700063fec78a8798001178ddcf1cf26fc1a77ed92aa8e5265d52f8ab2 2024-04-28T12:48:29.496 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 12:48:29 smithi171 systemd[1]: Started Ceph mds.f for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:29 smithi129 ceph-mon[62418]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:29 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/100710004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:29 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:29 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:29.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:29 smithi106 ceph-mon[28081]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:29.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:29 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/100710004' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:29.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:29 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:29.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:29 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:30.360 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:30 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:30 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:30 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.171:6834/4221893526,v1:172.21.15.171:6835/4221893526] up:boot 2024-04-28T12:48:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:30 smithi171 ceph-mon[63127]: fsmap 6 up:standby 2024-04-28T12:48:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:30 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-28T12:48:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:30 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:30 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:30 smithi171 ceph-mon[63127]: Deploying daemon mds.c on smithi171 2024-04-28T12:48:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:30 smithi171 ceph-mon[63127]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:30 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:30 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:30 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.171:6834/4221893526,v1:172.21.15.171:6835/4221893526] up:boot 2024-04-28T12:48:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:30 smithi129 ceph-mon[62418]: fsmap 6 up:standby 2024-04-28T12:48:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:30 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-28T12:48:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:30 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:30 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:30 smithi129 ceph-mon[62418]: Deploying daemon mds.c on smithi171 2024-04-28T12:48:30.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:30 smithi129 ceph-mon[62418]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:30.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.577+0000 7f901fe00640 1 -- 172.21.15.106:0/3006156571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9020109f80 msgr2=0x7f902010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:30.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.577+0000 7f901fe00640 1 --2- 172.21.15.106:0/3006156571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9020109f80 0x7f902010a400 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9008009ca0 tx=0x7f900802a8c0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.577+0000 7f901fe00640 1 -- 172.21.15.106:0/3006156571 shutdown_connections 2024-04-28T12:48:30.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.577+0000 7f901fe00640 1 --2- 172.21.15.106:0/3006156571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f902010a940 0x7f90201111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.577+0000 7f901fe00640 1 --2- 172.21.15.106:0/3006156571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9020109f80 0x7f902010a400 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.577+0000 7f901fe00640 1 --2- 172.21.15.106:0/3006156571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9020108d90 0x7f90201091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.577+0000 7f901fe00640 1 -- 172.21.15.106:0/3006156571 >> 172.21.15.106:0/3006156571 conn(0x7f9020077d30 msgr2=0x7f902007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:30.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.577+0000 7f901fe00640 1 -- 172.21.15.106:0/3006156571 shutdown_connections 2024-04-28T12:48:30.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.577+0000 7f901fe00640 1 -- 172.21.15.106:0/3006156571 wait complete. 2024-04-28T12:48:30.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:30.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:30.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:30 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.171:6834/4221893526,v1:172.21.15.171:6835/4221893526] up:boot 2024-04-28T12:48:30.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:30 smithi106 ceph-mon[28081]: fsmap 6 up:standby 2024-04-28T12:48:30.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-28T12:48:30.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:30.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:30 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:30.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:30 smithi106 ceph-mon[28081]: Deploying daemon mds.c on smithi171 2024-04-28T12:48:30.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:30 smithi106 ceph-mon[28081]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:30.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.578+0000 7f901fe00640 1 Processor -- start 2024-04-28T12:48:30.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f901fe00640 1 -- start start 2024-04-28T12:48:30.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f901fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9020108d90 0x7f902019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f901fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9020109f80 0x7f902019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f901fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f902010a940 0x7f90201a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f901fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f90201148c0 con 0x7f9020109f80 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f901fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9020114740 con 0x7f9020108d90 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f901fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9020114a40 con 0x7f902010a940 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f9016000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9020109f80 0x7f902019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f901ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9020108d90 0x7f902019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f901f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f902010a940 0x7f90201a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f9016000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9020109f80 0x7f902019f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33750/0 (socket says 172.21.15.106:33750) 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.579+0000 7f9016000640 1 -- 172.21.15.106:0/52667534 learned_addr learned my addr 172.21.15.106:0/52667534 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f901f400640 1 -- 172.21.15.106:0/52667534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9020108d90 msgr2=0x7f902019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f901f400640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9020108d90 0x7f902019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f901f400640 1 -- 172.21.15.106:0/52667534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9020109f80 msgr2=0x7f902019f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:30.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f901f400640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9020109f80 0x7f902019f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f901f400640 1 -- 172.21.15.106:0/52667534 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90201a6b90 con 0x7f902010a940 2024-04-28T12:48:30.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f901f400640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f902010a940 0x7f90201a6520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f901800e970 tx=0x7f901800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:30.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f9017e00640 1 -- 172.21.15.106:0/52667534 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901800ccc0 con 0x7f902010a940 2024-04-28T12:48:30.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f9017e00640 1 -- 172.21.15.106:0/52667534 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9018005020 con 0x7f902010a940 2024-04-28T12:48:30.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f9017e00640 1 -- 172.21.15.106:0/52667534 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9018004a00 con 0x7f902010a940 2024-04-28T12:48:30.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90201a6e80 con 0x7f902010a940 2024-04-28T12:48:30.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.580+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90201a72f0 con 0x7f902010a940 2024-04-28T12:48:30.578 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.582+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9020110130 con 0x7f902010a940 2024-04-28T12:48:30.579 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.583+0000 7f9017e00640 1 -- 172.21.15.106:0/52667534 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9018004ba0 con 0x7f902010a940 2024-04-28T12:48:30.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.585+0000 7f9017e00640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90040a3bb0 0x7f90040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:30.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.585+0000 7f9017e00640 1 -- 172.21.15.106:0/52667534 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f90180cfea0 con 0x7f902010a940 2024-04-28T12:48:30.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.585+0000 7f901ea00640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90040a3bb0 0x7f90040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:30.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.586+0000 7f901ea00640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90040a3bb0 0x7f90040a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9020109de0 tx=0x7f9010005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:30.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.589+0000 7f9017e00640 1 -- 172.21.15.106:0/52667534 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f901809cd20 con 0x7f902010a940 2024-04-28T12:48:30.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.830+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f902002dcd0 con 0x7f902010a940 2024-04-28T12:48:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.831+0000 7f9017e00640 1 -- 172.21.15.106:0/52667534 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5536 (secure 0 0 0) 0x7f901809c620 con 0x7f902010a940 2024-04-28T12:48:30.827 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:30.827 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-28T12:48:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2024-04-28T12:48:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90040a3bb0 msgr2=0x7f90040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90040a3bb0 0x7f90040a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9020109de0 tx=0x7f9010005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f902010a940 msgr2=0x7f90201a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f902010a940 0x7f90201a6520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f901800e970 tx=0x7f901800ee30 comp rx=0 tx=0).stop 2024-04-28T12:48:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 shutdown_connections 2024-04-28T12:48:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90040a3bb0 0x7f90040a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9020109f80 0x7f902019f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f902010a940 0x7f90201a6520 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 --2- 172.21.15.106:0/52667534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9020108d90 0x7f902019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:30.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 >> 172.21.15.106:0/52667534 conn(0x7f9020077d30 msgr2=0x7f902010d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:30.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 shutdown_connections 2024-04-28T12:48:30.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:30.834+0000 7f901fe00640 1 -- 172.21.15.106:0/52667534 wait complete. 2024-04-28T12:48:31.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:31 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/52667534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:31 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/52667534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:31.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:31 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/52667534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:31.672 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 12:48:31 smithi171 systemd[1]: Starting Ceph mds.c for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:48:31.949 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 12:48:31 smithi171 podman[93085]: 2024-04-28 12:48:31.579525468 +0000 UTC m=+0.018621580 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:31.949 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 12:48:31 smithi171 podman[93085]: 2024-04-28 12:48:31.690870576 +0000 UTC m=+0.129966681 container create d34769c4a5f88deca785c2378dae3ec6270d93151cd136cbef6524e425df7a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-28T12:48:31.949 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 12:48:31 smithi171 podman[93085]: 2024-04-28 12:48:31.954332221 +0000 UTC m=+0.393428327 container init d34769c4a5f88deca785c2378dae3ec6270d93151cd136cbef6524e425df7a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-28T12:48:32.245 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 12:48:31 smithi171 podman[93085]: 2024-04-28 12:48:31.956940496 +0000 UTC m=+0.396036612 container start d34769c4a5f88deca785c2378dae3ec6270d93151cd136cbef6524e425df7a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424) 2024-04-28T12:48:32.245 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 12:48:31 smithi171 ceph-mds[93103]: -- 172.21.15.171:0/2342041939 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5622622d9a40 con 0x5622622ac800 2024-04-28T12:48:32.245 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 12:48:31 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c[93099]: starting mds.c at 2024-04-28T12:48:32.245 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 12:48:32 smithi171 bash[93085]: d34769c4a5f88deca785c2378dae3ec6270d93151cd136cbef6524e425df7a90 2024-04-28T12:48:32.245 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 12:48:32 smithi171 systemd[1]: Started Ceph mds.c for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:32.851 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:33.063 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.067+0000 7f1a5a000640 1 -- 172.21.15.106:0/3436512236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a54100f50 msgr2=0x7f1a54108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.067+0000 7f1a5a000640 1 --2- 172.21.15.106:0/3436512236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a54100f50 0x7f1a54108150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009900 tx=0x7f1a3c02a400 comp rx=0 tx=0).stop 2024-04-28T12:48:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.067+0000 7f1a5a000640 1 -- 172.21.15.106:0/3436512236 shutdown_connections 2024-04-28T12:48:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.067+0000 7f1a5a000640 1 --2- 172.21.15.106:0/3436512236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a54108690 0x7f1a5410aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.067+0000 7f1a5a000640 1 --2- 172.21.15.106:0/3436512236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a54100f50 0x7f1a54108150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.067+0000 7f1a5a000640 1 --2- 172.21.15.106:0/3436512236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a54100580 0x7f1a54100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.067+0000 7f1a5a000640 1 -- 172.21.15.106:0/3436512236 >> 172.21.15.106:0/3436512236 conn(0x7f1a540fc120 msgr2=0x7f1a540fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.068+0000 7f1a5a000640 1 -- 172.21.15.106:0/3436512236 shutdown_connections 2024-04-28T12:48:33.064 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.068+0000 7f1a5a000640 1 -- 172.21.15.106:0/3436512236 wait complete. 2024-04-28T12:48:33.066 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a5a000640 1 Processor -- start 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a5a000640 1 -- start start 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a5a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a54100580 0x7f1a5419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a5a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a54100f50 0x7f1a5419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a5a000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a54108690 0x7f1a541a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a5a000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5410e470 con 0x7f1a54100580 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a5a000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5410e2f0 con 0x7f1a54108690 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a5a000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5410e5f0 con 0x7f1a54100f50 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a53400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a54100580 0x7f1a5419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a53400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a54100580 0x7f1a5419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33756/0 (socket says 172.21.15.106:33756) 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a53400640 1 -- 172.21.15.106:0/4291306958 learned_addr learned my addr 172.21.15.106:0/4291306958 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a4aa00640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a54100f50 0x7f1a5419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.069+0000 7f1a53e00640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a54108690 0x7f1a541a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:33.067 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a53400640 1 -- 172.21.15.106:0/4291306958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a54100f50 msgr2=0x7f1a5419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a53400640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a54100f50 0x7f1a5419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a53400640 1 -- 172.21.15.106:0/4291306958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a54108690 msgr2=0x7f1a541a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a53400640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a54108690 0x7f1a541a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a53400640 1 -- 172.21.15.106:0/4291306958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a541a29b0 con 0x7f1a54100580 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a53400640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a54100580 0x7f1a5419adf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400d950 tx=0x7f1a4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a50c00640 1 -- 172.21.15.106:0/4291306958 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44017070 con 0x7f1a54100580 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a50c00640 1 -- 172.21.15.106:0/4291306958 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a44009c30 con 0x7f1a54100580 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a541a2ca0 con 0x7f1a54100580 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a50c00640 1 -- 172.21.15.106:0/4291306958 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400f620 con 0x7f1a54100580 2024-04-28T12:48:33.068 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.070+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a541a31a0 con 0x7f1a54100580 2024-04-28T12:48:33.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.072+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a541a3370 con 0x7f1a54100580 2024-04-28T12:48:33.069 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.074+0000 7f1a50c00640 1 -- 172.21.15.106:0/4291306958 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1a44010460 con 0x7f1a54100580 2024-04-28T12:48:33.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.075+0000 7f1a50c00640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a380a3bb0 0x7f1a380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:33.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.075+0000 7f1a50c00640 1 -- 172.21.15.106:0/4291306958 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f1a440cfc30 con 0x7f1a54100580 2024-04-28T12:48:33.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.075+0000 7f1a4aa00640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a380a3bb0 0x7f1a380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:33.071 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.076+0000 7f1a4aa00640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a380a3bb0 0x7f1a380a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c000c00 tx=0x7f1a3c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:33.075 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.080+0000 7f1a50c00640 1 -- 172.21.15.106:0/4291306958 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1a44009de0 con 0x7f1a54100580 2024-04-28T12:48:33.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.323+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a541a35e0 con 0x7f1a54100580 2024-04-28T12:48:33.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.324+0000 7f1a50c00640 1 -- 172.21.15.106:0/4291306958 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7f1a4409c3c0 con 0x7f1a54100580 2024-04-28T12:48:33.320 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:33.320 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-28T12:48:33.321 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 8 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a380a3bb0 msgr2=0x7f1a380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a380a3bb0 0x7f1a380a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c000c00 tx=0x7f1a3c0314a0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a54100580 msgr2=0x7f1a5419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a54100580 0x7f1a5419adf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400d950 tx=0x7f1a4400de10 comp rx=0 tx=0).stop 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 shutdown_connections 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a380a3bb0 0x7f1a380a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a54108690 0x7f1a541a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a54100f50 0x7f1a5419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 --2- 172.21.15.106:0/4291306958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a54100580 0x7f1a5419adf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:33.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 >> 172.21.15.106:0/4291306958 conn(0x7f1a540fc120 msgr2=0x7f1a540fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 shutdown_connections 2024-04-28T12:48:33.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:33.327+0000 7f1a5a000640 1 -- 172.21.15.106:0/4291306958 wait complete. 2024-04-28T12:48:33.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:33.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:33.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:33.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:33.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:33.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:33.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:33.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: Deploying daemon mds.h on smithi129 2024-04-28T12:48:33.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] up:boot 2024-04-28T12:48:33.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: fsmap 7 up:standby 2024-04-28T12:48:33.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:33 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-28T12:48:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: Deploying daemon mds.h on smithi129 2024-04-28T12:48:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] up:boot 2024-04-28T12:48:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: fsmap 7 up:standby 2024-04-28T12:48:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:33 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-28T12:48:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: Deploying daemon mds.h on smithi129 2024-04-28T12:48:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] up:boot 2024-04-28T12:48:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: fsmap 7 up:standby 2024-04-28T12:48:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:33 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-28T12:48:34.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:34 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/4291306958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:34.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:34 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/4291306958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:34.513 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 12:48:34 smithi129 systemd[1]: Starting Ceph mds.h for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:48:34.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:34 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4291306958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:34.839 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 12:48:34 smithi129 podman[92551]: 2024-04-28 12:48:34.436205244 +0000 UTC m=+0.018148332 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:34.839 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 12:48:34 smithi129 podman[92551]: 2024-04-28 12:48:34.549641533 +0000 UTC m=+0.131584656 container create 4b77b95139ba44ccf8f39ca0e629069d2df9b3ae1742a39f367513887e746ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-28T12:48:35.125 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 12:48:34 smithi129 podman[92551]: 2024-04-28 12:48:34.844127924 +0000 UTC m=+0.426071004 container init 4b77b95139ba44ccf8f39ca0e629069d2df9b3ae1742a39f367513887e746ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-28T12:48:35.125 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 12:48:34 smithi129 podman[92551]: 2024-04-28 12:48:34.846737459 +0000 UTC m=+0.428680544 container start 4b77b95139ba44ccf8f39ca0e629069d2df9b3ae1742a39f367513887e746ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-28T12:48:35.125 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 12:48:34 smithi129 ceph-mds[92570]: -- 172.21.15.129:0/2334990938 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55622af6da40 con 0x55622af40800 2024-04-28T12:48:35.125 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 12:48:34 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h[92566]: starting mds.h at 2024-04-28T12:48:35.126 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 12:48:34 smithi129 bash[92551]: 4b77b95139ba44ccf8f39ca0e629069d2df9b3ae1742a39f367513887e746ebb 2024-04-28T12:48:35.126 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 12:48:34 smithi129 systemd[1]: Started Ceph mds.h for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:35.342 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:35.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:35 smithi106 ceph-mon[28081]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:35.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:35.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:35.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:35.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:35.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:35.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:35 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:35 smithi171 ceph-mon[63127]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:35.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:35 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:35 smithi129 ceph-mon[62418]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:35.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:35 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:35.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.559+0000 7f77d0600640 1 -- 172.21.15.106:0/3835093759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 msgr2=0x7f77c81113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:35.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.559+0000 7f77d0600640 1 --2- 172.21.15.106:0/3835093759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 0x7f77c81113d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009ca0 tx=0x7f77c002a8c0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.559+0000 7f77d0600640 1 -- 172.21.15.106:0/3835093759 shutdown_connections 2024-04-28T12:48:35.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.559+0000 7f77d0600640 1 --2- 172.21.15.106:0/3835093759 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 0x7f77c81113d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.559+0000 7f77d0600640 1 --2- 172.21.15.106:0/3835093759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c810a180 0x7f77c810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.559+0000 7f77d0600640 1 --2- 172.21.15.106:0/3835093759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77c8108f90 0x7f77c81093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.559+0000 7f77d0600640 1 -- 172.21.15.106:0/3835093759 >> 172.21.15.106:0/3835093759 conn(0x7f77c8077f40 msgr2=0x7f77c807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:35.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.560+0000 7f77d0600640 1 -- 172.21.15.106:0/3835093759 shutdown_connections 2024-04-28T12:48:35.556 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.560+0000 7f77d0600640 1 -- 172.21.15.106:0/3835093759 wait complete. 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.560+0000 7f77d0600640 1 Processor -- start 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77d0600640 1 -- start start 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77d0600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77c8108f90 0x7f77c819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77d0600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c810a180 0x7f77c819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77d0600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 0x7f77c81a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77d0600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f77c8114b20 con 0x7f77c8108f90 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77d0600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77c81149a0 con 0x7f77c810a180 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77d0600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f77c8114ca0 con 0x7f77c810ab40 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77ce600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 0x7f77c81a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:35.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77ce600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 0x7f77c81a66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:33160/0 (socket says 172.21.15.106:33160) 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77ce600640 1 -- 172.21.15.106:0/2293539476 learned_addr learned my addr 172.21.15.106:0/2293539476 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77cdc00640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77c8108f90 0x7f77c819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.561+0000 7f77cd200640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c810a180 0x7f77c819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77ce600640 1 -- 172.21.15.106:0/2293539476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c810a180 msgr2=0x7f77c819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77ce600640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c810a180 0x7f77c819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77ce600640 1 -- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77c8108f90 msgr2=0x7f77c819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77ce600640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77c8108f90 0x7f77c819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77ce600640 1 -- 172.21.15.106:0/2293539476 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c81a6d60 con 0x7f77c810ab40 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77cd200640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c810a180 0x7f77c819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77ce600640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 0x7f77c81a66f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f77c0033040 tx=0x7f77c00316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77c6000640 1 -- 172.21.15.106:0/2293539476 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0003b70 con 0x7f77c810ab40 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77c6000640 1 -- 172.21.15.106:0/2293539476 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c0003d10 con 0x7f77c810ab40 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77c6000640 1 -- 172.21.15.106:0/2293539476 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c002fb40 con 0x7f77c810ab40 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c81a6ff0 con 0x7f77c810ab40 2024-04-28T12:48:35.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.562+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c81a7460 con 0x7f77c810ab40 2024-04-28T12:48:35.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.564+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c8110250 con 0x7f77c810ab40 2024-04-28T12:48:35.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.565+0000 7f77c6000640 1 -- 172.21.15.106:0/2293539476 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f77c0004610 con 0x7f77c810ab40 2024-04-28T12:48:35.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.567+0000 7f77c6000640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f77b00a3bb0 0x7f77b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.567+0000 7f77c6000640 1 -- 172.21.15.106:0/2293539476 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f77c00f13c0 con 0x7f77c810ab40 2024-04-28T12:48:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.567+0000 7f77cdc00640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f77b00a3bb0 0x7f77b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:35.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.567+0000 7f77cdc00640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f77b00a3bb0 0x7f77b00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f77c8109fe0 tx=0x7f77bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:35.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.571+0000 7f77c6000640 1 -- 172.21.15.106:0/2293539476 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f77c00be1a0 con 0x7f77c810ab40 2024-04-28T12:48:35.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.816+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77c802dad0 con 0x7f77c810ab40 2024-04-28T12:48:35.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.817+0000 7f77c6000640 1 -- 172.21.15.106:0/2293539476 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7f77c00bdaa0 con 0x7f77c810ab40 2024-04-28T12:48:35.815 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:35.815 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-28T12:48:35.815 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 9 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.820+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f77b00a3bb0 msgr2=0x7f77b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.820+0000 7f77d0600640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f77b00a3bb0 0x7f77b00a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f77c8109fe0 tx=0x7f77bc005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.820+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 msgr2=0x7f77c81a66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.820+0000 7f77d0600640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 0x7f77c81a66f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f77c0033040 tx=0x7f77c00316f0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.821+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 shutdown_connections 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.821+0000 7f77d0600640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f77b00a3bb0 0x7f77b00a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.821+0000 7f77d0600640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f77c810ab40 0x7f77c81a66f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.821+0000 7f77d0600640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c810a180 0x7f77c819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.821+0000 7f77d0600640 1 --2- 172.21.15.106:0/2293539476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77c8108f90 0x7f77c819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.821+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 >> 172.21.15.106:0/2293539476 conn(0x7f77c8077f40 msgr2=0x7f77c810d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.821+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 shutdown_connections 2024-04-28T12:48:35.817 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:35.821+0000 7f77d0600640 1 -- 172.21.15.106:0/2293539476 wait complete. 2024-04-28T12:48:36.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:36 smithi171 ceph-mon[63127]: Deploying daemon mds.i on smithi171 2024-04-28T12:48:36.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:36 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.129:6836/1065884016,v1:172.21.15.129:6837/1065884016] up:boot 2024-04-28T12:48:36.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:36 smithi171 ceph-mon[63127]: fsmap 8 up:standby 2024-04-28T12:48:36.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:36 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-28T12:48:36.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:36 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2293539476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:36.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:36 smithi129 ceph-mon[62418]: Deploying daemon mds.i on smithi171 2024-04-28T12:48:36.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:36 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.129:6836/1065884016,v1:172.21.15.129:6837/1065884016] up:boot 2024-04-28T12:48:36.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:36 smithi129 ceph-mon[62418]: fsmap 8 up:standby 2024-04-28T12:48:36.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:36 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-28T12:48:36.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:36 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2293539476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:36.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:36 smithi106 ceph-mon[28081]: Deploying daemon mds.i on smithi171 2024-04-28T12:48:36.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:36 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.129:6836/1065884016,v1:172.21.15.129:6837/1065884016] up:boot 2024-04-28T12:48:36.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:36 smithi106 ceph-mon[28081]: fsmap 8 up:standby 2024-04-28T12:48:36.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:36 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-28T12:48:36.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:36 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2293539476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:37.204 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:37 smithi171 ceph-mon[63127]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:37.478 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:37 smithi106 ceph-mon[28081]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:37.495 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 12:48:37 smithi171 systemd[1]: Starting Ceph mds.i for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:48:37.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:37 smithi129 ceph-mon[62418]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:37.837 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:37.878 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 12:48:37 smithi171 podman[93391]: 2024-04-28 12:48:37.516987049 +0000 UTC m=+0.018669555 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:37.878 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 12:48:37 smithi171 podman[93391]: 2024-04-28 12:48:37.637477675 +0000 UTC m=+0.139160173 container create 56b0f9f78324d910e7b36b975b5c0888963f77a69d6e105511202d5539a31e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-28T12:48:37.878 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 12:48:37 smithi171 podman[93391]: 2024-04-28 12:48:37.857145609 +0000 UTC m=+0.358828106 container init 56b0f9f78324d910e7b36b975b5c0888963f77a69d6e105511202d5539a31e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T12:48:37.878 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 12:48:37 smithi171 podman[93391]: 2024-04-28 12:48:37.859845311 +0000 UTC m=+0.361527812 container start 56b0f9f78324d910e7b36b975b5c0888963f77a69d6e105511202d5539a31e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T12:48:37.878 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 12:48:37 smithi171 ceph-mds[93411]: -- 172.21.15.171:0/1100689340 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574a8f9da40 con 0x5574a8f70800 2024-04-28T12:48:38.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.047+0000 7f35bbe00640 1 -- 172.21.15.106:0/2782694432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35bc104320 msgr2=0x7f35bc10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:38.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.047+0000 7f35bbe00640 1 --2- 172.21.15.106:0/2782694432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35bc104320 0x7f35bc10abb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009be0 tx=0x7f35b402a6c0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.047+0000 7f35bbe00640 1 -- 172.21.15.106:0/2782694432 shutdown_connections 2024-04-28T12:48:38.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.047+0000 7f35bbe00640 1 --2- 172.21.15.106:0/2782694432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35bc104320 0x7f35bc10abb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.047+0000 7f35bbe00640 1 --2- 172.21.15.106:0/2782694432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35bc103960 0x7f35bc103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.047+0000 7f35bbe00640 1 --2- 172.21.15.106:0/2782694432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35bc102770 0x7f35bc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.047+0000 7f35bbe00640 1 -- 172.21.15.106:0/2782694432 >> 172.21.15.106:0/2782694432 conn(0x7f35bc0fdcc0 msgr2=0x7f35bc100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:38.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.047+0000 7f35bbe00640 1 -- 172.21.15.106:0/2782694432 shutdown_connections 2024-04-28T12:48:38.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.047+0000 7f35bbe00640 1 -- 172.21.15.106:0/2782694432 wait complete. 2024-04-28T12:48:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.048+0000 7f35bbe00640 1 Processor -- start 2024-04-28T12:48:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bbe00640 1 -- start start 2024-04-28T12:48:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35bc102770 0x7f35bc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bbe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35bc103960 0x7f35bc198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35bc104320 0x7f35bc19fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bbe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc10e280 con 0x7f35bc104320 2024-04-28T12:48:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bbe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc10e100 con 0x7f35bc103960 2024-04-28T12:48:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bbe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc10e400 con 0x7f35bc102770 2024-04-28T12:48:38.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35bc104320 0x7f35bc19fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35bc104320 0x7f35bc19fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33786/0 (socket says 172.21.15.106:33786) 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.049+0000 7f35bb400640 1 -- 172.21.15.106:0/1133023434 learned_addr learned my addr 172.21.15.106:0/1133023434 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.050+0000 7f35baa00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35bc102770 0x7f35bc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.050+0000 7f35ba000640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35bc103960 0x7f35bc198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.050+0000 7f35bb400640 1 -- 172.21.15.106:0/1133023434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35bc102770 msgr2=0x7f35bc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.050+0000 7f35bb400640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35bc102770 0x7f35bc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.050+0000 7f35bb400640 1 -- 172.21.15.106:0/1133023434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35bc103960 msgr2=0x7f35bc198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.050+0000 7f35bb400640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35bc103960 0x7f35bc198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.050+0000 7f35bb400640 1 -- 172.21.15.106:0/1133023434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc1a0550 con 0x7f35bc104320 2024-04-28T12:48:38.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.050+0000 7f35bb400640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35bc104320 0x7f35bc19fee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35b4034040 tx=0x7f35b4030f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:38.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.051+0000 7f35c0a00640 1 -- 172.21.15.106:0/1133023434 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4041020 con 0x7f35bc104320 2024-04-28T12:48:38.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.051+0000 7f35c0a00640 1 -- 172.21.15.106:0/1133023434 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b40318f0 con 0x7f35bc104320 2024-04-28T12:48:38.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.051+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35bc1a07e0 con 0x7f35bc104320 2024-04-28T12:48:38.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.051+0000 7f35c0a00640 1 -- 172.21.15.106:0/1133023434 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4036040 con 0x7f35bc104320 2024-04-28T12:48:38.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.051+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35bc1a0d20 con 0x7f35bc104320 2024-04-28T12:48:38.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.052+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35bc109a30 con 0x7f35bc104320 2024-04-28T12:48:38.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.054+0000 7f35c0a00640 1 -- 172.21.15.106:0/1133023434 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f35b4044020 con 0x7f35bc104320 2024-04-28T12:48:38.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.055+0000 7f35c0a00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35a00a3c30 0x7f35a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:38.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.055+0000 7f35c0a00640 1 -- 172.21.15.106:0/1133023434 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f35b40f0c20 con 0x7f35bc104320 2024-04-28T12:48:38.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.056+0000 7f35baa00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35a00a3c30 0x7f35a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:38.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.056+0000 7f35baa00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35a00a3c30 0x7f35a00a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009530 tx=0x7f35ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:38.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.060+0000 7f35c0a00640 1 -- 172.21.15.106:0/1133023434 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f35b404b020 con 0x7f35bc104320 2024-04-28T12:48:38.245 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 12:48:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i[93407]: starting mds.i at 2024-04-28T12:48:38.245 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 12:48:37 smithi171 bash[93391]: 56b0f9f78324d910e7b36b975b5c0888963f77a69d6e105511202d5539a31e15 2024-04-28T12:48:38.245 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 12:48:37 smithi171 systemd[1]: Started Ceph mds.i for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:38.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.310+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35bc02dad0 con 0x7f35bc104320 2024-04-28T12:48:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.312+0000 7f35c0a00640 1 -- 172.21.15.106:0/1133023434 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7f35b40bda00 con 0x7f35bc104320 2024-04-28T12:48:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:38.308 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-28T12:48:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 10 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35a00a3c30 msgr2=0x7f35a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35a00a3c30 0x7f35a00a60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009530 tx=0x7f35ac009260 comp rx=0 tx=0).stop 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35bc104320 msgr2=0x7f35bc19fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35bc104320 0x7f35bc19fee0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35b4034040 tx=0x7f35b4030f60 comp rx=0 tx=0).stop 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 shutdown_connections 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35a00a3c30 0x7f35a00a60f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35bc104320 0x7f35bc19fee0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35bc103960 0x7f35bc198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 --2- 172.21.15.106:0/1133023434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35bc102770 0x7f35bc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 >> 172.21.15.106:0/1133023434 conn(0x7f35bc0fdcc0 msgr2=0x7f35bc106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 shutdown_connections 2024-04-28T12:48:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:38.315+0000 7f35bbe00640 1 -- 172.21.15.106:0/1133023434 wait complete. 2024-04-28T12:48:39.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:39.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:39.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:39.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:39.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:39.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:39.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:39.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: Deploying daemon mds.g on smithi106 2024-04-28T12:48:39.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] up:boot 2024-04-28T12:48:39.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: fsmap 9 up:standby 2024-04-28T12:48:39.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-28T12:48:39.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:39 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1133023434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:39.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: Deploying daemon mds.g on smithi106 2024-04-28T12:48:39.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] up:boot 2024-04-28T12:48:39.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: fsmap 9 up:standby 2024-04-28T12:48:39.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-28T12:48:39.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:39 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1133023434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: Deploying daemon mds.g on smithi106 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] up:boot 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: fsmap 9 up:standby 2024-04-28T12:48:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-28T12:48:39.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:39 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1133023434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:40.331 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.825+0000 7ff52fc00640 1 -- 172.21.15.106:0/2006583425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5281033d0 msgr2=0x7ff5281057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.825+0000 7ff52fc00640 1 --2- 172.21.15.106:0/2006583425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5281033d0 0x7ff5281057c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009900 tx=0x7ff51c02a410 comp rx=0 tx=0).stop 2024-04-28T12:48:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.825+0000 7ff52fc00640 1 -- 172.21.15.106:0/2006583425 shutdown_connections 2024-04-28T12:48:40.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.825+0000 7ff52fc00640 1 --2- 172.21.15.106:0/2006583425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff528105d00 0x7ff5281080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:40.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.825+0000 7ff52fc00640 1 --2- 172.21.15.106:0/2006583425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff528108630 0x7ff52810aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:40.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.825+0000 7ff52fc00640 1 --2- 172.21.15.106:0/2006583425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5281033d0 0x7ff5281057c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:40.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.825+0000 7ff52fc00640 1 -- 172.21.15.106:0/2006583425 >> 172.21.15.106:0/2006583425 conn(0x7ff5280fcdc0 msgr2=0x7ff5280ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:40.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.825+0000 7ff52fc00640 1 -- 172.21.15.106:0/2006583425 shutdown_connections 2024-04-28T12:48:40.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.825+0000 7ff52fc00640 1 -- 172.21.15.106:0/2006583425 wait complete. 2024-04-28T12:48:40.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.827+0000 7ff52fc00640 1 Processor -- start 2024-04-28T12:48:40.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.827+0000 7ff52fc00640 1 -- start start 2024-04-28T12:48:40.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.827+0000 7ff52fc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5281033d0 0x7ff528198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:40.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.827+0000 7ff52fc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528105d00 0x7ff5281990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:40.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.827+0000 7ff52fc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff528108630 0x7ff5281a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:40.824 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.827+0000 7ff52fc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff52810e4f0 con 0x7ff528108630 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.827+0000 7ff52fc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff52810e370 con 0x7ff5281033d0 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.827+0000 7ff52fc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff52810e670 con 0x7ff528105d00 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff527e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528105d00 0x7ff5281990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff52dc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff528108630 0x7ff5281a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff527e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528105d00 0x7ff5281990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:51932/0 (socket says 172.21.15.106:51932) 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff527e00640 1 -- 172.21.15.106:0/3536103011 learned_addr learned my addr 172.21.15.106:0/3536103011 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff52d200640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5281033d0 0x7ff528198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff527e00640 1 -- 172.21.15.106:0/3536103011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5281033d0 msgr2=0x7ff528198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff527e00640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5281033d0 0x7ff528198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff527e00640 1 -- 172.21.15.106:0/3536103011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff528108630 msgr2=0x7ff5281a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff527e00640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff528108630 0x7ff5281a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff527e00640 1 -- 172.21.15.106:0/3536103011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5281a07b0 con 0x7ff528105d00 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.828+0000 7ff527e00640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528105d00 0x7ff5281990d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff514005fd0 tx=0x7ff51400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.829+0000 7ff525600640 1 -- 172.21.15.106:0/3536103011 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514017070 con 0x7ff528105d00 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.829+0000 7ff525600640 1 -- 172.21.15.106:0/3536103011 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff514009c30 con 0x7ff528105d00 2024-04-28T12:48:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.829+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5281a0aa0 con 0x7ff528105d00 2024-04-28T12:48:40.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.829+0000 7ff525600640 1 -- 172.21.15.106:0/3536103011 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514003d30 con 0x7ff528105d00 2024-04-28T12:48:40.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.829+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5281a1040 con 0x7ff528105d00 2024-04-28T12:48:40.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.830+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5280fe8b0 con 0x7ff528105d00 2024-04-28T12:48:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.833+0000 7ff525600640 1 -- 172.21.15.106:0/3536103011 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff514003ed0 con 0x7ff528105d00 2024-04-28T12:48:40.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.835+0000 7ff525600640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5100a3c30 0x7ff5100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:40.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.836+0000 7ff525600640 1 -- 172.21.15.106:0/3536103011 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff5140cfbb0 con 0x7ff528105d00 2024-04-28T12:48:40.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.836+0000 7ff52d200640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5100a3c30 0x7ff5100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:40.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.836+0000 7ff52d200640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5100a3c30 0x7ff5100a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff51c031500 tx=0x7ff51c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:40.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:40.839+0000 7ff525600640 1 -- 172.21.15.106:0/3536103011 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff514020030 con 0x7ff528105d00 2024-04-28T12:48:41.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.084+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff528063560 con 0x7ff528105d00 2024-04-28T12:48:41.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.085+0000 7ff525600640 1 -- 172.21.15.106:0/3536103011 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7ff514009de0 con 0x7ff528105d00 2024-04-28T12:48:41.081 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:41.081 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-28T12:48:41.081 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 10 2024-04-28T12:48:41.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5100a3c30 msgr2=0x7ff5100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:41.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5100a3c30 0x7ff5100a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff51c031500 tx=0x7ff51c005a20 comp rx=0 tx=0).stop 2024-04-28T12:48:41.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528105d00 msgr2=0x7ff5281990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:41.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528105d00 0x7ff5281990d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff514005fd0 tx=0x7ff51400dde0 comp rx=0 tx=0).stop 2024-04-28T12:48:41.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 shutdown_connections 2024-04-28T12:48:41.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5100a3c30 0x7ff5100a60f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:41.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff528108630 0x7ff5281a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:41.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528105d00 0x7ff5281990d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:41.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 --2- 172.21.15.106:0/3536103011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5281033d0 0x7ff528198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:41.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 >> 172.21.15.106:0/3536103011 conn(0x7ff5280fcdc0 msgr2=0x7ff528103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:41.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 shutdown_connections 2024-04-28T12:48:41.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:41.087+0000 7ff52fc00640 1 -- 172.21.15.106:0/3536103011 wait complete. 2024-04-28T12:48:41.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:41 smithi106 ceph-mon[28081]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:41.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:41 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3536103011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:41.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:41 smithi171 ceph-mon[63127]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:41.353 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:41 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3536103011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:41.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:41 smithi129 ceph-mon[62418]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:41.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:41 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3536103011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:41.719 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 12:48:41 smithi106 podman[68894]: 2024-04-28 12:48:41.426798764 +0000 UTC m=+0.155181877 container create 9adac8f9ea2c77ba618516f274ca94c684bc792819420723fc8c9069b0472ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-28T12:48:41.719 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 12:48:41 smithi106 podman[68894]: 2024-04-28 12:48:41.722170244 +0000 UTC m=+0.450553361 container init 9adac8f9ea2c77ba618516f274ca94c684bc792819420723fc8c9069b0472ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-28T12:48:42.074 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 12:48:41 smithi106 podman[68894]: 2024-04-28 12:48:41.731588198 +0000 UTC m=+0.459971276 container start 9adac8f9ea2c77ba618516f274ca94c684bc792819420723fc8c9069b0472ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux ) 2024-04-28T12:48:42.074 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 12:48:41 smithi106 ceph-mds[68913]: -- 172.21.15.106:0/504906277 <== mon.0 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618f90b7a40 con 0x5618f908a800 2024-04-28T12:48:42.074 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 12:48:41 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g[68909]: starting mds.g at 2024-04-28T12:48:42.074 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 12:48:41 smithi106 bash[68894]: 9adac8f9ea2c77ba618516f274ca94c684bc792819420723fc8c9069b0472ebb 2024-04-28T12:48:42.075 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 12:48:41 smithi106 systemd[1]: Started Ceph mds.g for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:43.103 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:43.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:43.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:43.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:43.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:43.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:43.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:43.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: Deploying daemon mds.a on smithi106 2024-04-28T12:48:43.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:43.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] up:boot 2024-04-28T12:48:43.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: fsmap 10 up:standby 2024-04-28T12:48:43.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:42 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-28T12:48:43.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:43.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:43.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:43.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:43.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:43.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:43.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: Deploying daemon mds.a on smithi106 2024-04-28T12:48:43.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:43.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] up:boot 2024-04-28T12:48:43.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: fsmap 10 up:standby 2024-04-28T12:48:43.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:42 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-28T12:48:43.265 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:43.265 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:43.265 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:43.265 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:43.265 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:43.265 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:43.265 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: Deploying daemon mds.a on smithi106 2024-04-28T12:48:43.265 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:43.266 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] up:boot 2024-04-28T12:48:43.266 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: fsmap 10 up:standby 2024-04-28T12:48:43.266 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:42 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-28T12:48:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.325+0000 7f4b33e00640 1 -- 172.21.15.106:0/3867371321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2c106720 msgr2=0x7f4b2c10cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.325+0000 7f4b33e00640 1 --2- 172.21.15.106:0/3867371321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2c106720 0x7f4b2c10cfb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009ca0 tx=0x7f4b2402a8c0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.325+0000 7f4b33e00640 1 -- 172.21.15.106:0/3867371321 shutdown_connections 2024-04-28T12:48:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.325+0000 7f4b33e00640 1 --2- 172.21.15.106:0/3867371321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2c106720 0x7f4b2c10cfb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.325+0000 7f4b33e00640 1 --2- 172.21.15.106:0/3867371321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c105d60 0x7f4b2c1061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.325+0000 7f4b33e00640 1 --2- 172.21.15.106:0/3867371321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2c104b70 0x7f4b2c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.325+0000 7f4b33e00640 1 -- 172.21.15.106:0/3867371321 >> 172.21.15.106:0/3867371321 conn(0x7f4b2c100120 msgr2=0x7f4b2c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.325+0000 7f4b33e00640 1 -- 172.21.15.106:0/3867371321 shutdown_connections 2024-04-28T12:48:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.325+0000 7f4b33e00640 1 -- 172.21.15.106:0/3867371321 wait complete. 2024-04-28T12:48:43.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.326+0000 7f4b33e00640 1 Processor -- start 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.326+0000 7f4b33e00640 1 -- start start 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b33e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2c104b70 0x7f4b2c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b33e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c105d60 0x7f4b2c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b33e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2c106720 0x7f4b2c1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b33e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1106d0 con 0x7f4b2c106720 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b33e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c110550 con 0x7f4b2c104b70 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b33e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c110850 con 0x7f4b2c105d60 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2c106720 0x7f4b2c1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b31e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2c106720 0x7f4b2c1a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36294/0 (socket says 172.21.15.106:36294) 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b31e00640 1 -- 172.21.15.106:0/63597700 learned_addr learned my addr 172.21.15.106:0/63597700 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b31400640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2c104b70 0x7f4b2c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b30a00640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c105d60 0x7f4b2c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b31400640 1 -- 172.21.15.106:0/63597700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c105d60 msgr2=0x7f4b2c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b31400640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c105d60 0x7f4b2c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b31400640 1 -- 172.21.15.106:0/63597700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2c106720 msgr2=0x7f4b2c1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b31400640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2c106720 0x7f4b2c1a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.327+0000 7f4b31400640 1 -- 172.21.15.106:0/63597700 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c1a2940 con 0x7f4b2c104b70 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.328+0000 7f4b31400640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2c104b70 0x7f4b2c19ad30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000e9c0 tx=0x7f4b2000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.328+0000 7f4b2a000640 1 -- 172.21.15.106:0/63597700 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000cd80 con 0x7f4b2c104b70 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.328+0000 7f4b2a000640 1 -- 172.21.15.106:0/63597700 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b200108e0 con 0x7f4b2c104b70 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.328+0000 7f4b2a000640 1 -- 172.21.15.106:0/63597700 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20018900 con 0x7f4b2c104b70 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.328+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c1a2c30 con 0x7f4b2c104b70 2024-04-28T12:48:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.328+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c1a3180 con 0x7f4b2c104b70 2024-04-28T12:48:43.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.329+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2c10be90 con 0x7f4b2c104b70 2024-04-28T12:48:43.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.331+0000 7f4b2a000640 1 -- 172.21.15.106:0/63597700 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4b2001f020 con 0x7f4b2c104b70 2024-04-28T12:48:43.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.332+0000 7f4b2a000640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4b140a3bb0 0x7f4b140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:43.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.332+0000 7f4b2a000640 1 -- 172.21.15.106:0/63597700 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4b200cfff0 con 0x7f4b2c104b70 2024-04-28T12:48:43.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.333+0000 7f4b30a00640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4b140a3bb0 0x7f4b140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:43.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.333+0000 7f4b30a00640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4b140a3bb0 0x7f4b140a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b180095f0 tx=0x7f4b18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:43.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.337+0000 7f4b2a000640 1 -- 172.21.15.106:0/63597700 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4b2009cdf0 con 0x7f4b2c104b70 2024-04-28T12:48:43.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.586+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b2c063560 con 0x7f4b2c104b70 2024-04-28T12:48:43.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.587+0000 7f4b2a000640 1 -- 172.21.15.106:0/63597700 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8848 (secure 0 0 0) 0x7f4b2009c6f0 con 0x7f4b2c104b70 2024-04-28T12:48:43.583 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:43.584 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-28T12:48:43.584 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 11 2024-04-28T12:48:43.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.590+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4b140a3bb0 msgr2=0x7f4b140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:43.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.590+0000 7f4b33e00640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4b140a3bb0 0x7f4b140a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4b180095f0 tx=0x7f4b18005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.590+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2c104b70 msgr2=0x7f4b2c19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.590+0000 7f4b33e00640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2c104b70 0x7f4b2c19ad30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4b2000e9c0 tx=0x7f4b2000ee80 comp rx=0 tx=0).stop 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.590+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 shutdown_connections 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.590+0000 7f4b33e00640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4b140a3bb0 0x7f4b140a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.590+0000 7f4b33e00640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b2c106720 0x7f4b2c1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.591+0000 7f4b33e00640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c105d60 0x7f4b2c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.591+0000 7f4b33e00640 1 --2- 172.21.15.106:0/63597700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b2c104b70 0x7f4b2c19ad30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.591+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 >> 172.21.15.106:0/63597700 conn(0x7f4b2c100120 msgr2=0x7f4b2c109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.591+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 shutdown_connections 2024-04-28T12:48:43.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:43.591+0000 7f4b33e00640 1 -- 172.21.15.106:0/63597700 wait complete. 2024-04-28T12:48:44.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:44 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/63597700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:44.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:44 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/63597700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:44.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:44 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/63597700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:45.184 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:45 smithi106 ceph-mon[28081]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:45 smithi171 ceph-mon[63127]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:45.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:45 smithi129 ceph-mon[62418]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:45.555 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 12:48:45 smithi106 systemd[1]: Starting Ceph mds.a for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:48:45.607 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:45.815 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 12:48:45 smithi106 podman[69236]: 2024-04-28 12:48:45.558440245 +0000 UTC m=+0.125887930 container create 03e72f101791c9d60f36560856413f2296f99b81685ead4386bd8edbcd843252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-28T12:48:45.815 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 12:48:45 smithi106 podman[69236]: 2024-04-28 12:48:45.467164877 +0000 UTC m=+0.034612524 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:45.815 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 12:48:45 smithi106 podman[69236]: 2024-04-28 12:48:45.820078875 +0000 UTC m=+0.387526533 container init 03e72f101791c9d60f36560856413f2296f99b81685ead4386bd8edbcd843252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T12:48:45.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.822+0000 7f6384600640 1 -- 172.21.15.106:0/1138541906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c105d80 msgr2=0x7f637c106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:45.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.822+0000 7f6384600640 1 --2- 172.21.15.106:0/1138541906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c105d80 0x7f637c106200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6368009900 tx=0x7f636802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:48:45.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.822+0000 7f6384600640 1 -- 172.21.15.106:0/1138541906 shutdown_connections 2024-04-28T12:48:45.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.822+0000 7f6384600640 1 --2- 172.21.15.106:0/1138541906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f637c106740 0x7f637c10cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:45.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.822+0000 7f6384600640 1 --2- 172.21.15.106:0/1138541906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c105d80 0x7f637c106200 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:45.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.822+0000 7f6384600640 1 --2- 172.21.15.106:0/1138541906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f637c104b90 0x7f637c104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:45.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.822+0000 7f6384600640 1 -- 172.21.15.106:0/1138541906 >> 172.21.15.106:0/1138541906 conn(0x7f637c100120 msgr2=0x7f637c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:45.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.823+0000 7f6384600640 1 -- 172.21.15.106:0/1138541906 shutdown_connections 2024-04-28T12:48:45.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.823+0000 7f6384600640 1 -- 172.21.15.106:0/1138541906 wait complete. 2024-04-28T12:48:45.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.824+0000 7f6384600640 1 Processor -- start 2024-04-28T12:48:45.819 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.824+0000 7f6384600640 1 -- start start 2024-04-28T12:48:45.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.824+0000 7f6384600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f637c104b90 0x7f637c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:45.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.824+0000 7f6384600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c105d80 0x7f637c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:45.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.824+0000 7f6384600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f637c106740 0x7f637c1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:45.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.824+0000 7f6384600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f637c1106f0 con 0x7f637c106740 2024-04-28T12:48:45.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.824+0000 7f6384600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f637c110570 con 0x7f637c105d80 2024-04-28T12:48:45.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.824+0000 7f6384600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f637c110870 con 0x7f637c104b90 2024-04-28T12:48:45.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c105d80 0x7f637c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:45.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f637c104b90 0x7f637c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:45.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f637c104b90 0x7f637c19ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:51980/0 (socket says 172.21.15.106:51980) 2024-04-28T12:48:45.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381c00640 1 -- 172.21.15.106:0/1917290681 learned_addr learned my addr 172.21.15.106:0/1917290681 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:45.821 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6382600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f637c106740 0x7f637c1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:45.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381c00640 1 -- 172.21.15.106:0/1917290681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c105d80 msgr2=0x7f637c19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:45.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381c00640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c105d80 0x7f637c19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:45.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381c00640 1 -- 172.21.15.106:0/1917290681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f637c106740 msgr2=0x7f637c1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:45.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381c00640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f637c106740 0x7f637c1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:45.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381c00640 1 -- 172.21.15.106:0/1917290681 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c1a2980 con 0x7f637c104b90 2024-04-28T12:48:45.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.825+0000 7f6381200640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c105d80 0x7f637c19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:48:45.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.826+0000 7f6381c00640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f637c104b90 0x7f637c19ad60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f637000e9c0 tx=0x7f637000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:45.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.826+0000 7f637a000640 1 -- 172.21.15.106:0/1917290681 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637000cde0 con 0x7f637c104b90 2024-04-28T12:48:45.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.826+0000 7f637a000640 1 -- 172.21.15.106:0/1917290681 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63700108e0 con 0x7f637c104b90 2024-04-28T12:48:45.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.826+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637c1a2c70 con 0x7f637c104b90 2024-04-28T12:48:45.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.826+0000 7f637a000640 1 -- 172.21.15.106:0/1917290681 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370018980 con 0x7f637c104b90 2024-04-28T12:48:45.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.826+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637c1a3190 con 0x7f637c104b90 2024-04-28T12:48:45.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.829+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f637c10beb0 con 0x7f637c104b90 2024-04-28T12:48:45.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.830+0000 7f637a000640 1 -- 172.21.15.106:0/1917290681 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f637001f020 con 0x7f637c104b90 2024-04-28T12:48:45.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.831+0000 7f637a000640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f63640a3c30 0x7f63640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:45.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.831+0000 7f637a000640 1 -- 172.21.15.106:0/1917290681 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f63700d0120 con 0x7f637c104b90 2024-04-28T12:48:45.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.831+0000 7f6381200640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f63640a3c30 0x7f63640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:45.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.832+0000 7f6381200640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f63640a3c30 0x7f63640a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6368000c00 tx=0x7f63680314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:45.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:45.835+0000 7f637a000640 1 -- 172.21.15.106:0/1917290681 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f637009cf00 con 0x7f637c104b90 2024-04-28T12:48:46.074 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 12:48:45 smithi106 podman[69236]: 2024-04-28 12:48:45.82954481 +0000 UTC m=+0.396992458 container start 03e72f101791c9d60f36560856413f2296f99b81685ead4386bd8edbcd843252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T12:48:46.075 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 12:48:45 smithi106 ceph-mds[69294]: -- 172.21.15.106:0/422386183 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e85f29da40 con 0x55e85f270c00 2024-04-28T12:48:46.075 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 12:48:45 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a[69267]: starting mds.a at 2024-04-28T12:48:46.075 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 12:48:45 smithi106 bash[69236]: 03e72f101791c9d60f36560856413f2296f99b81685ead4386bd8edbcd843252 2024-04-28T12:48:46.075 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 12:48:45 smithi106 systemd[1]: Started Ceph mds.a for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:46.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.106+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f637c02dad0 con 0x7f637c104b90 2024-04-28T12:48:46.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.108+0000 7f637a000640 1 -- 172.21.15.106:0/1917290681 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9671 (secure 0 0 0) 0x7f637009c800 con 0x7f637c104b90 2024-04-28T12:48:46.104 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:46.104 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-28T12:48:46.104 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 12 2024-04-28T12:48:46.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f63640a3c30 msgr2=0x7f63640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:46.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f63640a3c30 0x7f63640a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6368000c00 tx=0x7f63680314c0 comp rx=0 tx=0).stop 2024-04-28T12:48:46.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f637c104b90 msgr2=0x7f637c19ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:46.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f637c104b90 0x7f637c19ad60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f637000e9c0 tx=0x7f637000ee80 comp rx=0 tx=0).stop 2024-04-28T12:48:46.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 shutdown_connections 2024-04-28T12:48:46.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f63640a3c30 0x7f63640a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:46.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f637c106740 0x7f637c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:46.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f637c105d80 0x7f637c19b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:46.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 --2- 172.21.15.106:0/1917290681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f637c104b90 0x7f637c19ad60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:46.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.110+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 >> 172.21.15.106:0/1917290681 conn(0x7f637c100120 msgr2=0x7f637c109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:46.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.111+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 shutdown_connections 2024-04-28T12:48:46.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:46.111+0000 7f6384600640 1 -- 172.21.15.106:0/1917290681 wait complete. 2024-04-28T12:48:47.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] up:boot 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: fsmap 11 up:standby 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1917290681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:47.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:47 smithi106 ceph-mon[28081]: Deploying daemon mds.b on smithi129 2024-04-28T12:48:47.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:47.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] up:boot 2024-04-28T12:48:47.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: fsmap 11 up:standby 2024-04-28T12:48:47.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-28T12:48:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1917290681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:47 smithi171 ceph-mon[63127]: Deploying daemon mds.b on smithi129 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] up:boot 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: fsmap 11 up:standby 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1917290681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-28T12:48:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:47.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:47 smithi129 ceph-mon[62418]: Deploying daemon mds.b on smithi129 2024-04-28T12:48:48.127 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:48.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.344+0000 7f8d9be00640 1 -- 172.21.15.106:0/2959527199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 msgr2=0x7f8d9c10cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:48.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.344+0000 7f8d9be00640 1 --2- 172.21.15.106:0/2959527199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 0x7f8d9c10cdc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009ca0 tx=0x7f8d9402a8c0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.344+0000 7f8d9be00640 1 -- 172.21.15.106:0/2959527199 shutdown_connections 2024-04-28T12:48:48.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.344+0000 7f8d9be00640 1 --2- 172.21.15.106:0/2959527199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d9c105b70 0x7f8d9c105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.344+0000 7f8d9be00640 1 --2- 172.21.15.106:0/2959527199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 0x7f8d9c10cdc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.344+0000 7f8d9be00640 1 --2- 172.21.15.106:0/2959527199 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d9c104980 0x7f8d9c104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.344+0000 7f8d9be00640 1 -- 172.21.15.106:0/2959527199 >> 172.21.15.106:0/2959527199 conn(0x7f8d9c0fff10 msgr2=0x7f8d9c102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:48.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.345+0000 7f8d9be00640 1 -- 172.21.15.106:0/2959527199 shutdown_connections 2024-04-28T12:48:48.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.345+0000 7f8d9be00640 1 -- 172.21.15.106:0/2959527199 wait complete. 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.346+0000 7f8d9be00640 1 Processor -- start 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.346+0000 7f8d9be00640 1 -- start start 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.346+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d9c104980 0x7f8d9c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.346+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d9c105b70 0x7f8d9c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.346+0000 7f8d9be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 0x7f8d9c1a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.346+0000 7f8d9be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c110500 con 0x7f8d9c104980 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.346+0000 7f8d9be00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c110380 con 0x7f8d9c106530 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.346+0000 7f8d9be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c110680 con 0x7f8d9c105b70 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 0x7f8d9c1a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d9c104980 0x7f8d9c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 0x7f8d9c1a2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:46130/0 (socket says 172.21.15.106:46130) 2024-04-28T12:48:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9b400640 1 -- 172.21.15.106:0/3030699405 learned_addr learned my addr 172.21.15.106:0/3030699405 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d93e00640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d9c105b70 0x7f8d9c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9b400640 1 -- 172.21.15.106:0/3030699405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d9c105b70 msgr2=0x7f8d9c19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9b400640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d9c105b70 0x7f8d9c19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9b400640 1 -- 172.21.15.106:0/3030699405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d9c104980 msgr2=0x7f8d9c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9b400640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d9c104980 0x7f8d9c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9b400640 1 -- 172.21.15.106:0/3030699405 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c1a2790 con 0x7f8d9c106530 2024-04-28T12:48:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.347+0000 7f8d9b400640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 0x7f8d9c1a2120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d94033040 tx=0x7f8d940316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.348+0000 7f8d93400640 1 -- 172.21.15.106:0/3030699405 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94003b70 con 0x7f8d9c106530 2024-04-28T12:48:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.348+0000 7f8d93400640 1 -- 172.21.15.106:0/3030699405 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d94003d10 con 0x7f8d9c106530 2024-04-28T12:48:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.348+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9c1a2a20 con 0x7f8d9c106530 2024-04-28T12:48:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.348+0000 7f8d93400640 1 -- 172.21.15.106:0/3030699405 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9402fb40 con 0x7f8d9c106530 2024-04-28T12:48:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.348+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9c1a2f70 con 0x7f8d9c106530 2024-04-28T12:48:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.349+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d9c10bca0 con 0x7f8d9c106530 2024-04-28T12:48:48.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.351+0000 7f8d93400640 1 -- 172.21.15.106:0/3030699405 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8d94004610 con 0x7f8d9c106530 2024-04-28T12:48:48.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.352+0000 7f8d93400640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d800a3bb0 0x7f8d800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:48.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.352+0000 7f8d93400640 1 -- 172.21.15.106:0/3030699405 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8d940f13c0 con 0x7f8d9c106530 2024-04-28T12:48:48.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.352+0000 7f8d9aa00640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d800a3bb0 0x7f8d800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:48.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.353+0000 7f8d9aa00640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d800a3bb0 0x7f8d800a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c1059d0 tx=0x7f8d8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:48.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.356+0000 7f8d93400640 1 -- 172.21.15.106:0/3030699405 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8d940be1a0 con 0x7f8d9c106530 2024-04-28T12:48:48.459 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:48:48 smithi129 systemd[1]: Starting Ceph mds.b for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T12:48:48.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.598+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d9c063560 con 0x7f8d9c106530 2024-04-28T12:48:48.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.600+0000 7f8d93400640 1 -- 172.21.15.106:0/3030699405 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9671 (secure 0 0 0) 0x7f8d940bdaa0 con 0x7f8d9c106530 2024-04-28T12:48:48.595 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:48.595 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-28T12:48:48.595 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 12 2024-04-28T12:48:48.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d800a3bb0 msgr2=0x7f8d800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:48.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d800a3bb0 0x7f8d800a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c1059d0 tx=0x7f8d8c005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:48.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 msgr2=0x7f8d9c1a2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:48.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 0x7f8d9c1a2120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d94033040 tx=0x7f8d940316f0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 shutdown_connections 2024-04-28T12:48:48.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d800a3bb0 0x7f8d800a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d9c106530 0x7f8d9c1a2120 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d9c105b70 0x7f8d9c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 --2- 172.21.15.106:0/3030699405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d9c104980 0x7f8d9c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:48.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.602+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 >> 172.21.15.106:0/3030699405 conn(0x7f8d9c0fff10 msgr2=0x7f8d9c108e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:48.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.603+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 shutdown_connections 2024-04-28T12:48:48.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:48.603+0000 7f8d9be00640 1 -- 172.21.15.106:0/3030699405 wait complete. 2024-04-28T12:48:48.711 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:48:48 smithi129 podman[92858]: 2024-04-28 12:48:48.366835673 +0000 UTC m=+0.018291259 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T12:48:48.711 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:48:48 smithi129 podman[92858]: 2024-04-28 12:48:48.480108916 +0000 UTC m=+0.131564553 container create 86d9c5e1c4dab9b8bf365cb8bde33604dfc3c585b13ed85a590c706da44d9408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T12:48:48.711 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:48:48 smithi129 podman[92858]: 2024-04-28 12:48:48.71608716 +0000 UTC m=+0.367542749 container init 86d9c5e1c4dab9b8bf365cb8bde33604dfc3c585b13ed85a590c706da44d9408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-28T12:48:49.013 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:48:48 smithi129 podman[92858]: 2024-04-28 12:48:48.718688356 +0000 UTC m=+0.370143945 container start 86d9c5e1c4dab9b8bf365cb8bde33604dfc3c585b13ed85a590c706da44d9408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-28T12:48:49.013 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:48:48 smithi129 ceph-mds[92877]: -- 172.21.15.129:0/3234243879 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d84ca71a40 con 0x55d84ca45000 2024-04-28T12:48:49.013 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:48:48 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: starting mds.b at 2024-04-28T12:48:49.013 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:48:48 smithi129 bash[92858]: 86d9c5e1c4dab9b8bf365cb8bde33604dfc3c585b13ed85a590c706da44d9408 2024-04-28T12:48:49.014 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:48:48 smithi129 systemd[1]: Started Ceph mds.b for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T12:48:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:49 smithi106 ceph-mon[28081]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:49 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3030699405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:49 smithi171 ceph-mon[63127]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:49 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3030699405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:49 smithi129 ceph-mon[62418]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:49.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:49 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3030699405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:49.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:49.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:50.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:50 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] up:boot 2024-04-28T12:48:50.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:50 smithi129 ceph-mon[62418]: fsmap 12 up:standby 2024-04-28T12:48:50.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:50 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-28T12:48:50.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:50 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:50.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:50 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] up:boot 2024-04-28T12:48:50.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:50 smithi106 ceph-mon[28081]: fsmap 12 up:standby 2024-04-28T12:48:50.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:50 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-28T12:48:50.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:50 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:50.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:50 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] up:boot 2024-04-28T12:48:50.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:50 smithi171 ceph-mon[63127]: fsmap 12 up:standby 2024-04-28T12:48:50.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:50 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-28T12:48:50.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:50 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:50.617 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:48:50.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.810+0000 7fa8c4e00640 1 -- 172.21.15.106:0/3211538957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0103b60 msgr2=0x7fa8c0103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:50.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.810+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/3211538957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0103b60 0x7fa8c0103fe0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009900 tx=0x7fa8a802a410 comp rx=0 tx=0).stop 2024-04-28T12:48:50.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.810+0000 7fa8c4e00640 1 -- 172.21.15.106:0/3211538957 shutdown_connections 2024-04-28T12:48:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.810+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/3211538957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0103b60 0x7fa8c0103fe0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.810+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/3211538957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8c0104520 0x7fa8c010adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.810+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/3211538957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8c0102970 0x7fa8c0102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.810+0000 7fa8c4e00640 1 -- 172.21.15.106:0/3211538957 >> 172.21.15.106:0/3211538957 conn(0x7fa8c00fdee0 msgr2=0x7fa8c0100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.810+0000 7fa8c4e00640 1 -- 172.21.15.106:0/3211538957 shutdown_connections 2024-04-28T12:48:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.810+0000 7fa8c4e00640 1 -- 172.21.15.106:0/3211538957 wait complete. 2024-04-28T12:48:50.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.811+0000 7fa8c4e00640 1 Processor -- start 2024-04-28T12:48:50.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.811+0000 7fa8c4e00640 1 -- start start 2024-04-28T12:48:50.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8c4e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0102970 0x7fa8c019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:50.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8c4e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8c0103b60 0x7fa8c019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8c4e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8c0104520 0x7fa8c01a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8c4e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c010e4a0 con 0x7fa8c0102970 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8c4e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c010e320 con 0x7fa8c0104520 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8c4e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c010e620 con 0x7fa8c0103b60 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8bde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0102970 0x7fa8c019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8bde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0102970 0x7fa8c019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48064/0 (socket says 172.21.15.106:48064) 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8bde00640 1 -- 172.21.15.106:0/1363173116 learned_addr learned my addr 172.21.15.106:0/1363173116 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8be800640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8c0104520 0x7fa8c01a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8b5400640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8c0103b60 0x7fa8c019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8bde00640 1 -- 172.21.15.106:0/1363173116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8c0103b60 msgr2=0x7fa8c019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8bde00640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8c0103b60 0x7fa8c019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8bde00640 1 -- 172.21.15.106:0/1363173116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8c0104520 msgr2=0x7fa8c01a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8bde00640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8c0104520 0x7fa8c01a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:50.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8bde00640 1 -- 172.21.15.106:0/1363173116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c01a29f0 con 0x7fa8c0102970 2024-04-28T12:48:50.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.812+0000 7fa8be800640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8c0104520 0x7fa8c01a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:50.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.813+0000 7fa8bde00640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0102970 0x7fa8c019add0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000e9c0 tx=0x7fa8b000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:50.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.813+0000 7fa8b7400640 1 -- 172.21.15.106:0/1363173116 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b000ccc0 con 0x7fa8c0102970 2024-04-28T12:48:50.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.813+0000 7fa8b7400640 1 -- 172.21.15.106:0/1363173116 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b00035f0 con 0x7fa8c0102970 2024-04-28T12:48:50.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.813+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c01a2ce0 con 0x7fa8c0102970 2024-04-28T12:48:50.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.813+0000 7fa8b7400640 1 -- 172.21.15.106:0/1363173116 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b000ce60 con 0x7fa8c0102970 2024-04-28T12:48:50.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.813+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c01a31b0 con 0x7fa8c0102970 2024-04-28T12:48:50.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.815+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c0069c10 con 0x7fa8c0102970 2024-04-28T12:48:50.812 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.816+0000 7fa8b7400640 1 -- 172.21.15.106:0/1363173116 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa8b0004000 con 0x7fa8c0102970 2024-04-28T12:48:50.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.818+0000 7fa8b7400640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa8a00a3bb0 0x7fa8a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.818+0000 7fa8b7400640 1 -- 172.21.15.106:0/1363173116 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa8b00d04d0 con 0x7fa8c0102970 2024-04-28T12:48:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.818+0000 7fa8b5400640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa8a00a3bb0 0x7fa8a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:50.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.819+0000 7fa8b5400640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa8a00a3bb0 0x7fa8a00a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8000c00 tx=0x7fa8a80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:50.818 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:50.823+0000 7fa8b7400640 1 -- 172.21.15.106:0/1363173116 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa8b009d2b0 con 0x7fa8c0102970 2024-04-28T12:48:51.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.091+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8c01a33d0 con 0x7fa8c0102970 2024-04-28T12:48:51.087 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.092+0000 7fa8b7400640 1 -- 172.21.15.106:0/1363173116 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10500 (secure 0 0 0) 0x7fa8b009cbb0 con 0x7fa8c0102970 2024-04-28T12:48:51.088 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:51.088 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24578,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-28T12:48:51.088 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 13 2024-04-28T12:48:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa8a00a3bb0 msgr2=0x7fa8a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa8a00a3bb0 0x7fa8a00a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8000c00 tx=0x7fa8a80314a0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0102970 msgr2=0x7fa8c019add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0102970 0x7fa8c019add0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000e9c0 tx=0x7fa8b000edc0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 shutdown_connections 2024-04-28T12:48:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa8a00a3bb0 0x7fa8a00a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8c0104520 0x7fa8c01a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8c0103b60 0x7fa8c019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 --2- 172.21.15.106:0/1363173116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c0102970 0x7fa8c019add0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.094+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 >> 172.21.15.106:0/1363173116 conn(0x7fa8c00fdee0 msgr2=0x7fa8c0109380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.095+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 shutdown_connections 2024-04-28T12:48:51.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.095+0000 7fa8c4e00640 1 -- 172.21.15.106:0/1363173116 wait complete. 2024-04-28T12:48:51.109 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:48:51.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:51 smithi171 ceph-mon[63127]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:51.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:51 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:51 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:51 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:51 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:51 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.314+0000 7f90afa00640 1 -- 172.21.15.106:0/2921820084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 msgr2=0x7f90a81061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.314+0000 7f90afa00640 1 --2- 172.21.15.106:0/2921820084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 0x7f90a81061e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9094009900 tx=0x7f909402a3f0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.315+0000 7f90afa00640 1 -- 172.21.15.106:0/2921820084 shutdown_connections 2024-04-28T12:48:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.315+0000 7f90afa00640 1 --2- 172.21.15.106:0/2921820084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 0x7f90a81061e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.315+0000 7f90afa00640 1 --2- 172.21.15.106:0/2921820084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a8106720 0x7f90a810cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.315+0000 7f90afa00640 1 --2- 172.21.15.106:0/2921820084 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a8104b70 0x7f90a8104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.315+0000 7f90afa00640 1 -- 172.21.15.106:0/2921820084 >> 172.21.15.106:0/2921820084 conn(0x7f90a8100120 msgr2=0x7f90a8102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.315+0000 7f90afa00640 1 -- 172.21.15.106:0/2921820084 shutdown_connections 2024-04-28T12:48:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.315+0000 7f90afa00640 1 -- 172.21.15.106:0/2921820084 wait complete. 2024-04-28T12:48:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90afa00640 1 Processor -- start 2024-04-28T12:48:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90afa00640 1 -- start start 2024-04-28T12:48:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90afa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a8104b70 0x7f90a819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90afa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 0x7f90a819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90afa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a8106720 0x7f90a81a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90afa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f90a81106d0 con 0x7f90a8105d60 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90afa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8110550 con 0x7f90a8106720 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90afa00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8110850 con 0x7f90a8104b70 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90a7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 0x7f90a819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90a7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 0x7f90a819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48076/0 (socket says 172.21.15.106:48076) 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90a7e00640 1 -- 172.21.15.106:0/808664294 learned_addr learned my addr 172.21.15.106:0/808664294 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90ada00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a8106720 0x7f90a81a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.316+0000 7f90ad000640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a8104b70 0x7f90a819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.317+0000 7f90a7e00640 1 -- 172.21.15.106:0/808664294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a8104b70 msgr2=0x7f90a819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.317+0000 7f90a7e00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a8104b70 0x7f90a819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.317+0000 7f90a7e00640 1 -- 172.21.15.106:0/808664294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a8106720 msgr2=0x7f90a81a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.317+0000 7f90a7e00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a8106720 0x7f90a81a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.317+0000 7f90a7e00640 1 -- 172.21.15.106:0/808664294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a81a29f0 con 0x7f90a8105d60 2024-04-28T12:48:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.317+0000 7f90ada00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a8106720 0x7f90a81a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.317+0000 7f90a7e00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 0x7f90a819b310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9094005ab0 tx=0x7f9094003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.317+0000 7f90a5600640 1 -- 172.21.15.106:0/808664294 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909403f070 con 0x7f90a8105d60 2024-04-28T12:48:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.317+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a81a2c80 con 0x7f90a8105d60 2024-04-28T12:48:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.318+0000 7f90a5600640 1 -- 172.21.15.106:0/808664294 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9094031c50 con 0x7f90a8105d60 2024-04-28T12:48:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.318+0000 7f90a5600640 1 -- 172.21.15.106:0/808664294 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9094035030 con 0x7f90a8105d60 2024-04-28T12:48:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.318+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a81a31c0 con 0x7f90a8105d60 2024-04-28T12:48:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.319+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a810beb0 con 0x7f90a8105d60 2024-04-28T12:48:51.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.321+0000 7f90a5600640 1 -- 172.21.15.106:0/808664294 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9094042070 con 0x7f90a8105d60 2024-04-28T12:48:51.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.322+0000 7f90a5600640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90900a3bb0 0x7f90900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:51.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.322+0000 7f90a5600640 1 -- 172.21.15.106:0/808664294 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f90940f0690 con 0x7f90a8105d60 2024-04-28T12:48:51.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.323+0000 7f90ad000640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90900a3bb0 0x7f90900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:51.319 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.323+0000 7f90ad000640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90900a3bb0 0x7f90900a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f90a8105bc0 tx=0x7f909c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:51.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.327+0000 7f90a5600640 1 -- 172.21.15.106:0/808664294 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f90940bd470 con 0x7f90a8105d60 2024-04-28T12:48:51.323 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:51 smithi106 ceph-mon[28081]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:51.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:51 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:51 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:51 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:51 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:51 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:51 smithi129 ceph-mon[62418]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:51 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:51 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:51 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:51 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:51 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:51.533 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.538+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90a8063530 con 0x7f90a8105d60 2024-04-28T12:48:51.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.539+0000 7f90a5600640 1 -- 172.21.15.106:0/808664294 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18524 (secure 0 0 0) 0x7f90940bcd70 con 0x7f90a8105d60 2024-04-28T12:48:51.535 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:51.535 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":81,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:47:59.467490+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90900a3bb0 msgr2=0x7f90900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90900a3bb0 0x7f90900a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f90a8105bc0 tx=0x7f909c005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 msgr2=0x7f90a819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 0x7f90a819b310 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9094005ab0 tx=0x7f9094003b90 comp rx=0 tx=0).stop 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 shutdown_connections 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90900a3bb0 0x7f90900a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90a8105d60 0x7f90a819b310 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90a8106720 0x7f90a81a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 --2- 172.21.15.106:0/808664294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90a8104b70 0x7f90a819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 >> 172.21.15.106:0/808664294 conn(0x7f90a8100120 msgr2=0x7f90a8109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 shutdown_connections 2024-04-28T12:48:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.542+0000 7f90afa00640 1 -- 172.21.15.106:0/808664294 wait complete. 2024-04-28T12:48:51.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-28T12:48:51.561 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:48:51.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.751+0000 7f04eb200640 1 -- 172.21.15.106:0/129541223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec101740 msgr2=0x7f04ec101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.752+0000 7f04eb200640 1 --2- 172.21.15.106:0/129541223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec101740 0x7f04ec101b60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009900 tx=0x7f04dc02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.752+0000 7f04eb200640 1 -- 172.21.15.106:0/129541223 shutdown_connections 2024-04-28T12:48:51.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.752+0000 7f04eb200640 1 --2- 172.21.15.106:0/129541223 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04ec1032f0 0x7f04ec109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.752+0000 7f04eb200640 1 --2- 172.21.15.106:0/129541223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04ec102930 0x7f04ec102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.752+0000 7f04eb200640 1 --2- 172.21.15.106:0/129541223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec101740 0x7f04ec101b60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.752+0000 7f04eb200640 1 -- 172.21.15.106:0/129541223 >> 172.21.15.106:0/129541223 conn(0x7f04ec0fccd0 msgr2=0x7f04ec0ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:51.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.752+0000 7f04eb200640 1 -- 172.21.15.106:0/129541223 shutdown_connections 2024-04-28T12:48:51.748 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.752+0000 7f04eb200640 1 -- 172.21.15.106:0/129541223 wait complete. 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04eb200640 1 Processor -- start 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04eb200640 1 -- start start 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04eb200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04ec101740 0x7f04ec198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04eb200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04ec102930 0x7f04ec199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04eb200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec1032f0 0x7f04ec1a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04eb200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec10d2c0 con 0x7f04ec1032f0 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04eb200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec10d140 con 0x7f04ec101740 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04eb200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec10d440 con 0x7f04ec102930 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04ea800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec1032f0 0x7f04ec1a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04e9400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04ec102930 0x7f04ec199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04ea800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec1032f0 0x7f04ec1a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48098/0 (socket says 172.21.15.106:48098) 2024-04-28T12:48:51.751 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04ea800640 1 -- 172.21.15.106:0/3740928873 learned_addr learned my addr 172.21.15.106:0/3740928873 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.753+0000 7f04e9e00640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04ec101740 0x7f04ec198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04e9400640 1 -- 172.21.15.106:0/3740928873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04ec101740 msgr2=0x7f04ec198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04e9400640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04ec101740 0x7f04ec198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04e9400640 1 -- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec1032f0 msgr2=0x7f04ec1a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04e9400640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec1032f0 0x7f04ec1a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04e9400640 1 -- 172.21.15.106:0/3740928873 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec1a0690 con 0x7f04ec102930 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04ea800640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec1032f0 0x7f04ec1a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04e9400640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04ec102930 0x7f04ec199020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005fd0 tx=0x7f04d800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04e2a00640 1 -- 172.21.15.106:0/3740928873 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8009c30 con 0x7f04ec102930 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04e2a00640 1 -- 172.21.15.106:0/3740928873 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d8009dd0 con 0x7f04ec102930 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ec1a0980 con 0x7f04ec102930 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.754+0000 7f04e2a00640 1 -- 172.21.15.106:0/3740928873 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d8010460 con 0x7f04ec102930 2024-04-28T12:48:51.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.755+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ec1a0ed0 con 0x7f04ec102930 2024-04-28T12:48:51.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.756+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ec1a1100 con 0x7f04ec102930 2024-04-28T12:48:51.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.758+0000 7f04e2a00640 1 -- 172.21.15.106:0/3740928873 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f04d8003e80 con 0x7f04ec102930 2024-04-28T12:48:51.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.759+0000 7f04e2a00640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f04d00a3bb0 0x7f04d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:51.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.760+0000 7f04e2a00640 1 -- 172.21.15.106:0/3740928873 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f04d80cff30 con 0x7f04ec102930 2024-04-28T12:48:51.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.760+0000 7f04e9e00640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f04d00a3bb0 0x7f04d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:51.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.760+0000 7f04e9e00640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f04d00a3bb0 0x7f04d00a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005ab0 tx=0x7f04dc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:51.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:51.764+0000 7f04e2a00640 1 -- 172.21.15.106:0/3740928873 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f04d8017020 con 0x7f04ec102930 2024-04-28T12:48:51.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.002+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04ec1a1370 con 0x7f04ec102930 2024-04-28T12:48:51.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.003+0000 7f04e2a00640 1 -- 172.21.15.106:0/3740928873 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f04d809c6c0 con 0x7f04ec102930 2024-04-28T12:48:51.999 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T12:48:52.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.005+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f04d00a3bb0 msgr2=0x7f04d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.001 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.005+0000 7f04eb200640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f04d00a3bb0 0x7f04d00a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005ab0 tx=0x7f04dc005a20 comp rx=0 tx=0).stop 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.005+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04ec102930 msgr2=0x7f04ec199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.005+0000 7f04eb200640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04ec102930 0x7f04ec199020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005fd0 tx=0x7f04d800dde0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.005+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 shutdown_connections 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.005+0000 7f04eb200640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f04d00a3bb0 0x7f04d00a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.005+0000 7f04eb200640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04ec1032f0 0x7f04ec1a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.005+0000 7f04eb200640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04ec102930 0x7f04ec199020 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.006+0000 7f04eb200640 1 --2- 172.21.15.106:0/3740928873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f04ec101740 0x7f04ec198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.006+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 >> 172.21.15.106:0/3740928873 conn(0x7f04ec0fccd0 msgr2=0x7f04ec0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.006+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 shutdown_connections 2024-04-28T12:48:52.002 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.006+0000 7f04eb200640 1 -- 172.21.15.106:0/3740928873 wait complete. 2024-04-28T12:48:52.020 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-28T12:48:52.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.212+0000 7fe21c600640 1 -- 172.21.15.106:0/3099409533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214104bb0 msgr2=0x7fe214104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.212+0000 7fe21c600640 1 --2- 172.21.15.106:0/3099409533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214104bb0 0x7fe214104fd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe208009900 tx=0x7fe20802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.212+0000 7fe21c600640 1 -- 172.21.15.106:0/3099409533 shutdown_connections 2024-04-28T12:48:52.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.212+0000 7fe21c600640 1 --2- 172.21.15.106:0/3099409533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214106760 0x7fe21410cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.212+0000 7fe21c600640 1 --2- 172.21.15.106:0/3099409533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe214105da0 0x7fe214106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.212+0000 7fe21c600640 1 --2- 172.21.15.106:0/3099409533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214104bb0 0x7fe214104fd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.212+0000 7fe21c600640 1 -- 172.21.15.106:0/3099409533 >> 172.21.15.106:0/3099409533 conn(0x7fe214100120 msgr2=0x7fe214102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:52.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.213+0000 7fe21c600640 1 -- 172.21.15.106:0/3099409533 shutdown_connections 2024-04-28T12:48:52.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.213+0000 7fe21c600640 1 -- 172.21.15.106:0/3099409533 wait complete. 2024-04-28T12:48:52.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.214+0000 7fe21c600640 1 Processor -- start 2024-04-28T12:48:52.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.214+0000 7fe21c600640 1 -- start start 2024-04-28T12:48:52.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.214+0000 7fe21c600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214104bb0 0x7fe21419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:52.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.214+0000 7fe21c600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214105da0 0x7fe21419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:52.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.214+0000 7fe21c600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe214106760 0x7fe2141a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:52.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.214+0000 7fe21c600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe214110730 con 0x7fe214105da0 2024-04-28T12:48:52.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.214+0000 7fe21c600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141105b0 con 0x7fe214106760 2024-04-28T12:48:52.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.214+0000 7fe21c600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141108b0 con 0x7fe214104bb0 2024-04-28T12:48:52.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe219200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214105da0 0x7fe21419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe21a600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe214106760 0x7fe2141a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe219c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214104bb0 0x7fe21419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe219c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214104bb0 0x7fe21419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:51166/0 (socket says 172.21.15.106:51166) 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe219200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214105da0 0x7fe21419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48128/0 (socket says 172.21.15.106:48128) 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe21a600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe214106760 0x7fe2141a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44012/0 (socket says 172.21.15.106:44012) 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe219c00640 1 -- 172.21.15.106:0/1041881610 learned_addr learned my addr 172.21.15.106:0/1041881610 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe21a600640 1 -- 172.21.15.106:0/1041881610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214104bb0 msgr2=0x7fe21419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe21a600640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214104bb0 0x7fe21419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe21a600640 1 -- 172.21.15.106:0/1041881610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214105da0 msgr2=0x7fe21419b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe21a600640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214105da0 0x7fe21419b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe21a600640 1 -- 172.21.15.106:0/1041881610 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2141a2a20 con 0x7fe214106760 2024-04-28T12:48:52.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.215+0000 7fe21a600640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe214106760 0x7fe2141a23b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009fd0 tx=0x7fe20c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:52.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.216+0000 7fe212000640 1 -- 172.21.15.106:0/1041881610 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00ce20 con 0x7fe214106760 2024-04-28T12:48:52.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.216+0000 7fe212000640 1 -- 172.21.15.106:0/1041881610 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe20c005020 con 0x7fe214106760 2024-04-28T12:48:52.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.216+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2141a2d10 con 0x7fe214106760 2024-04-28T12:48:52.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.216+0000 7fe212000640 1 -- 172.21.15.106:0/1041881610 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c010690 con 0x7fe214106760 2024-04-28T12:48:52.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.216+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2141a3250 con 0x7fe214106760 2024-04-28T12:48:52.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.218+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21410bed0 con 0x7fe214106760 2024-04-28T12:48:52.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.219+0000 7fe212000640 1 -- 172.21.15.106:0/1041881610 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe20c0054d0 con 0x7fe214106760 2024-04-28T12:48:52.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.221+0000 7fe212000640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc0a3c30 0x7fe1fc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:52.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.221+0000 7fe212000640 1 -- 172.21.15.106:0/1041881610 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe20c0d05f0 con 0x7fe214106760 2024-04-28T12:48:52.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.221+0000 7fe219c00640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc0a3c30 0x7fe1fc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:52.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.222+0000 7fe219c00640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc0a3c30 0x7fe1fc0a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe208000c00 tx=0x7fe208005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:52.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.225+0000 7fe212000640 1 -- 172.21.15.106:0/1041881610 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe20c09d3d0 con 0x7fe214106760 2024-04-28T12:48:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:52 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1363173116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:52 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/808664294' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:48:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:52.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:52 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3740928873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:48:52.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.434+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe214063530 con 0x7fe214106760 2024-04-28T12:48:52.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.435+0000 7fe212000640 1 -- 172.21.15.106:0/1041881610 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fe20c09ccd0 con 0x7fe214106760 2024-04-28T12:48:52.434 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:48:52.434 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2024-04-28T12:48:52.434 INFO:teuthology.orchestra.run.smithi106.stdout: { 2024-04-28T12:48:52.434 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2024-04-28T12:48:52.434 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2024-04-28T12:48:52.434 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-28T12:48:52.434 INFO:teuthology.orchestra.run.smithi106.stdout:] 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc0a3c30 msgr2=0x7fe1fc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc0a3c30 0x7fe1fc0a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe208000c00 tx=0x7fe208005a20 comp rx=0 tx=0).stop 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe214106760 msgr2=0x7fe2141a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe214106760 0x7fe2141a23b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009fd0 tx=0x7fe20c00ef10 comp rx=0 tx=0).stop 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 shutdown_connections 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1fc0a3c30 0x7fe1fc0a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214105da0 0x7fe21419b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe214106760 0x7fe2141a23b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 --2- 172.21.15.106:0/1041881610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe214104bb0 0x7fe21419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 >> 172.21.15.106:0/1041881610 conn(0x7fe214100120 msgr2=0x7fe2141090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 shutdown_connections 2024-04-28T12:48:52.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.438+0000 7fe21c600640 1 -- 172.21.15.106:0/1041881610 wait complete. 2024-04-28T12:48:52.453 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-28T12:48:52.454 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-28T12:48:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:52 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1363173116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:52 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/808664294' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:48:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:52.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:52 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3740928873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:48:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:52 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1363173116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:48:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:52 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/808664294' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:48:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:52.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:52 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3740928873' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:48:52.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.649+0000 7f1465a00640 1 -- 172.21.15.106:0/2293010583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1460103b70 msgr2=0x7f1460103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.649+0000 7f1465a00640 1 --2- 172.21.15.106:0/2293010583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1460103b70 0x7f1460103ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f144c009900 tx=0x7f144c02a410 comp rx=0 tx=0).stop 2024-04-28T12:48:52.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.649+0000 7f1465a00640 1 -- 172.21.15.106:0/2293010583 shutdown_connections 2024-04-28T12:48:52.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.649+0000 7f1465a00640 1 --2- 172.21.15.106:0/2293010583 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1460104530 0x7f146010adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.649+0000 7f1465a00640 1 --2- 172.21.15.106:0/2293010583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1460103b70 0x7f1460103ff0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.649+0000 7f1465a00640 1 --2- 172.21.15.106:0/2293010583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1460102980 0x7f1460102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.649+0000 7f1465a00640 1 -- 172.21.15.106:0/2293010583 >> 172.21.15.106:0/2293010583 conn(0x7f14600fdf10 msgr2=0x7f1460100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:52.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.649+0000 7f1465a00640 1 -- 172.21.15.106:0/2293010583 shutdown_connections 2024-04-28T12:48:52.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.650+0000 7f1465a00640 1 -- 172.21.15.106:0/2293010583 wait complete. 2024-04-28T12:48:52.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.650+0000 7f1465a00640 1 Processor -- start 2024-04-28T12:48:52.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f1465a00640 1 -- start start 2024-04-28T12:48:52.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f1465a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1460102980 0x7f146019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:52.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f1465a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1460103b70 0x7f146019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:52.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f1465a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1460104530 0x7f14601a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:52.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f1465a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f146010e4b0 con 0x7f1460104530 2024-04-28T12:48:52.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f1465a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f146010e330 con 0x7f1460102980 2024-04-28T12:48:52.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f1465a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f146010e630 con 0x7f1460103b70 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f145e000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1460103b70 0x7f146019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f145ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1460102980 0x7f146019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f145e000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1460103b70 0x7f146019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:51178/0 (socket says 172.21.15.106:51178) 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f145e000640 1 -- 172.21.15.106:0/64974587 learned_addr learned my addr 172.21.15.106:0/64974587 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.651+0000 7f145f400640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1460104530 0x7f14601a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f145ea00640 1 -- 172.21.15.106:0/64974587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1460103b70 msgr2=0x7f146019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f145ea00640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1460103b70 0x7f146019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f145ea00640 1 -- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1460104530 msgr2=0x7f14601a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f145ea00640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1460104530 0x7f14601a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f145ea00640 1 -- 172.21.15.106:0/64974587 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14601a2a00 con 0x7f1460102980 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f145ea00640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1460102980 0x7f146019ade0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f145000e9c0 tx=0x7f145000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f1457400640 1 -- 172.21.15.106:0/64974587 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145000ccc0 con 0x7f1460102980 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14601a2cf0 con 0x7f1460102980 2024-04-28T12:48:52.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f1457400640 1 -- 172.21.15.106:0/64974587 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14500035f0 con 0x7f1460102980 2024-04-28T12:48:52.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f1457400640 1 -- 172.21.15.106:0/64974587 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145000ce60 con 0x7f1460102980 2024-04-28T12:48:52.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.652+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14601a31c0 con 0x7f1460102980 2024-04-28T12:48:52.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.654+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14601a33f0 con 0x7f1460102980 2024-04-28T12:48:52.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.655+0000 7f1457400640 1 -- 172.21.15.106:0/64974587 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1450004000 con 0x7f1460102980 2024-04-28T12:48:52.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.657+0000 7f1457400640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14440a3bb0 0x7f14440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:52.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.657+0000 7f1457400640 1 -- 172.21.15.106:0/64974587 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f14500d0430 con 0x7f1460102980 2024-04-28T12:48:52.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.657+0000 7f145e000640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14440a3bb0 0x7f14440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:52.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.658+0000 7f145e000640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14440a3bb0 0x7f14440a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f144c009230 tx=0x7f144c0058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:52.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.661+0000 7f1457400640 1 -- 172.21.15.106:0/64974587 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f145009d210 con 0x7f1460102980 2024-04-28T12:48:52.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:52.904+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f14601a37e0 con 0x7f1460102980 2024-04-28T12:48:53.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.089+0000 7f1457400640 1 -- 172.21.15.106:0/64974587 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f145009cb10 con 0x7f1460102980 2024-04-28T12:48:53.086 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2024-04-28T12:48:53.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14440a3bb0 msgr2=0x7f14440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:53.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14440a3bb0 0x7f14440a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f144c009230 tx=0x7f144c0058d0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1460102980 msgr2=0x7f146019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:53.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1460102980 0x7f146019ade0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f145000e9c0 tx=0x7f145000ee80 comp rx=0 tx=0).stop 2024-04-28T12:48:53.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 shutdown_connections 2024-04-28T12:48:53.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f14440a3bb0 0x7f14440a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1460104530 0x7f14601a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1460103b70 0x7f146019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 --2- 172.21.15.106:0/64974587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1460102980 0x7f146019ade0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.096+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 >> 172.21.15.106:0/64974587 conn(0x7f14600fdf10 msgr2=0x7f1460109340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:53.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.097+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 shutdown_connections 2024-04-28T12:48:53.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.097+0000 7f1465a00640 1 -- 172.21.15.106:0/64974587 wait complete. 2024-04-28T12:48:53.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-28T12:48:53.269 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:53 smithi171 ceph-mon[63127]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:53.269 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:53 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:53.269 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:53 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1041881610' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-28T12:48:53.269 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:53 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/64974587' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-28T12:48:53.269 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:53 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-28T12:48:53.285 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:53 smithi129 ceph-mon[62418]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:53.285 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:53 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:53.285 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:53 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1041881610' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-28T12:48:53.286 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:53 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/64974587' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-28T12:48:53.286 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:53 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-28T12:48:53.294 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.298+0000 7fd68be00640 1 -- 172.21.15.106:0/2640997245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd684103410 msgr2=0x7fd684105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:53.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.298+0000 7fd68be00640 1 --2- 172.21.15.106:0/2640997245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd684103410 0x7fd684105800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd67c009ca0 tx=0x7fd67c02a8c0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.299+0000 7fd68be00640 1 -- 172.21.15.106:0/2640997245 shutdown_connections 2024-04-28T12:48:53.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.299+0000 7fd68be00640 1 --2- 172.21.15.106:0/2640997245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd684105d40 0x7fd684108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.299+0000 7fd68be00640 1 --2- 172.21.15.106:0/2640997245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd684108670 0x7fd68410aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.299+0000 7fd68be00640 1 --2- 172.21.15.106:0/2640997245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd684103410 0x7fd684105800 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.299+0000 7fd68be00640 1 -- 172.21.15.106:0/2640997245 >> 172.21.15.106:0/2640997245 conn(0x7fd6840fce00 msgr2=0x7fd6840ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:53.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.299+0000 7fd68be00640 1 -- 172.21.15.106:0/2640997245 shutdown_connections 2024-04-28T12:48:53.295 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.299+0000 7fd68be00640 1 -- 172.21.15.106:0/2640997245 wait complete. 2024-04-28T12:48:53.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd68be00640 1 Processor -- start 2024-04-28T12:48:53.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd68be00640 1 -- start start 2024-04-28T12:48:53.296 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd68be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd684103410 0x7fd68419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd68be00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd684105d40 0x7fd68419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd68be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd684108670 0x7fd6841a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd68be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410e530 con 0x7fd684108670 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd68be00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410e3b0 con 0x7fd684105d40 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd68be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd68410e6b0 con 0x7fd684103410 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd689400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd684103410 0x7fd68419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd689e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd684108670 0x7fd6841a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd689400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd684103410 0x7fd68419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:51194/0 (socket says 172.21.15.106:51194) 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd689400640 1 -- 172.21.15.106:0/1483456306 learned_addr learned my addr 172.21.15.106:0/1483456306 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.300+0000 7fd688a00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd684105d40 0x7fd68419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd688a00640 1 -- 172.21.15.106:0/1483456306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd684103410 msgr2=0x7fd68419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd688a00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd684103410 0x7fd68419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd688a00640 1 -- 172.21.15.106:0/1483456306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd684108670 msgr2=0x7fd6841a23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd688a00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd684108670 0x7fd6841a23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:53.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd688a00640 1 -- 172.21.15.106:0/1483456306 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6841a2a20 con 0x7fd684105d40 2024-04-28T12:48:53.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd688a00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd684105d40 0x7fd68419b340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd67000e9c0 tx=0x7fd67000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:53.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd682000640 1 -- 172.21.15.106:0/1483456306 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67000ccc0 con 0x7fd684105d40 2024-04-28T12:48:53.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd682000640 1 -- 172.21.15.106:0/1483456306 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6700108e0 con 0x7fd684105d40 2024-04-28T12:48:53.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd682000640 1 -- 172.21.15.106:0/1483456306 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700187c0 con 0x7fd684105d40 2024-04-28T12:48:53.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.301+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6841a2d10 con 0x7fd684105d40 2024-04-28T12:48:53.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.302+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6841a3200 con 0x7fd684105d40 2024-04-28T12:48:53.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.303+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd68410a1c0 con 0x7fd684105d40 2024-04-28T12:48:53.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.305+0000 7fd682000640 1 -- 172.21.15.106:0/1483456306 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd670010430 con 0x7fd684105d40 2024-04-28T12:48:53.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.306+0000 7fd682000640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6680a3bb0 0x7fd6680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:53.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.307+0000 7fd682000640 1 -- 172.21.15.106:0/1483456306 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7fd6700d0430 con 0x7fd684105d40 2024-04-28T12:48:53.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.307+0000 7fd689400640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6680a3bb0 0x7fd6680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:53.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.307+0000 7fd689400640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6680a3bb0 0x7fd6680a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd67c005fe0 tx=0x7fd67c005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.310+0000 7fd682000640 1 -- 172.21.15.106:0/1483456306 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd67009ca50 con 0x7fd684105d40 2024-04-28T12:48:53.446 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:53 smithi106 ceph-mon[28081]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:53.446 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:53 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:48:53.446 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:53 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1041881610' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-28T12:48:53.446 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:53 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/64974587' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-28T12:48:53.446 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:53 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-28T12:48:53.512 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:53.516+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fd6840635d0 con 0x7fd684105d40 2024-04-28T12:48:54.104 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.108+0000 7fd682000640 1 -- 172.21.15.106:0/1483456306 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7fd6700a0d50 con 0x7fd684105d40 2024-04-28T12:48:54.105 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2024-04-28T12:48:54.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6680a3bb0 msgr2=0x7fd6680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:54.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6680a3bb0 0x7fd6680a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd67c005fe0 tx=0x7fd67c005f70 comp rx=0 tx=0).stop 2024-04-28T12:48:54.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd684105d40 msgr2=0x7fd68419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:54.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd684105d40 0x7fd68419b340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd67000e9c0 tx=0x7fd67000ee80 comp rx=0 tx=0).stop 2024-04-28T12:48:54.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 shutdown_connections 2024-04-28T12:48:54.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6680a3bb0 0x7fd6680a6070 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd67c005fe0 tx=0x7fd67c005f70 comp rx=0 tx=0).stop 2024-04-28T12:48:54.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd684108670 0x7fd6841a23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:54.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd684105d40 0x7fd68419b340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:54.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 --2- 172.21.15.106:0/1483456306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd684103410 0x7fd68419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:54.109 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.113+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 >> 172.21.15.106:0/1483456306 conn(0x7fd6840fce00 msgr2=0x7fd684102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:54.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.117+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 shutdown_connections 2024-04-28T12:48:54.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.117+0000 7fd68be00640 1 -- 172.21.15.106:0/1483456306 wait complete. 2024-04-28T12:48:54.141 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-28T12:48:54.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.329+0000 7f0832a00640 1 -- 172.21.15.106:0/172860555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f082c076910 msgr2=0x7f082c074d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:54.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.329+0000 7f0832a00640 1 --2- 172.21.15.106:0/172860555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f082c076910 0x7f082c074d60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f081c009ca0 tx=0x7f081c02a950 comp rx=0 tx=0).stop 2024-04-28T12:48:54.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.329+0000 7f0832a00640 1 -- 172.21.15.106:0/172860555 shutdown_connections 2024-04-28T12:48:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.329+0000 7f0832a00640 1 --2- 172.21.15.106:0/172860555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f082c075330 0x7f082c075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.329+0000 7f0832a00640 1 --2- 172.21.15.106:0/172860555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f082c10a810 0x7f082c10ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.329+0000 7f0832a00640 1 --2- 172.21.15.106:0/172860555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f082c076910 0x7f082c074d60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.329+0000 7f0832a00640 1 -- 172.21.15.106:0/172860555 >> 172.21.15.106:0/172860555 conn(0x7f082c0fdf10 msgr2=0x7f082c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.329+0000 7f0832a00640 1 -- 172.21.15.106:0/172860555 shutdown_connections 2024-04-28T12:48:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.330+0000 7f0832a00640 1 -- 172.21.15.106:0/172860555 wait complete. 2024-04-28T12:48:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.330+0000 7f0832a00640 1 Processor -- start 2024-04-28T12:48:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0832a00640 1 -- start start 2024-04-28T12:48:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0832a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f082c075330 0x7f082c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0832a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f082c076910 0x7f082c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0832a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f082c10a810 0x7f082c1a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0832a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f082c1106d0 con 0x7f082c10a810 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0832a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f082c110550 con 0x7f082c076910 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0832a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f082c110850 con 0x7f082c075330 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0830a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f082c10a810 0x7f082c1a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f082be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f082c075330 0x7f082c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0823400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f082c076910 0x7f082c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0830a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f082c10a810 0x7f082c1a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48180/0 (socket says 172.21.15.106:48180) 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.331+0000 7f0830a00640 1 -- 172.21.15.106:0/1765135634 learned_addr learned my addr 172.21.15.106:0/1765135634 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0823400640 1 -- 172.21.15.106:0/1765135634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f082c075330 msgr2=0x7f082c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0823400640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f082c075330 0x7f082c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0823400640 1 -- 172.21.15.106:0/1765135634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f082c10a810 msgr2=0x7f082c1a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0823400640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f082c10a810 0x7f082c1a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0823400640 1 -- 172.21.15.106:0/1765135634 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082c1a29f0 con 0x7f082c076910 2024-04-28T12:48:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f082be00640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f082c075330 0x7f082c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0823400640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f082c076910 0x7f082c19b310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f081400d6d0 tx=0x7f081400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0829600640 1 -- 172.21.15.106:0/1765135634 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814017070 con 0x7f082c076910 2024-04-28T12:48:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0829600640 1 -- 172.21.15.106:0/1765135634 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0814009820 con 0x7f082c076910 2024-04-28T12:48:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082c1a2ce0 con 0x7f082c076910 2024-04-28T12:48:54.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0829600640 1 -- 172.21.15.106:0/1765135634 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0814004c40 con 0x7f082c076910 2024-04-28T12:48:54.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.332+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082c1a3220 con 0x7f082c076910 2024-04-28T12:48:54.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.334+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082c1095d0 con 0x7f082c076910 2024-04-28T12:48:54.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.335+0000 7f0829600640 1 -- 172.21.15.106:0/1765135634 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0814005220 con 0x7f082c076910 2024-04-28T12:48:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.337+0000 7f0829600640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f08100a3c30 0x7f08100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.337+0000 7f0829600640 1 -- 172.21.15.106:0/1765135634 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f08140cfca0 con 0x7f082c076910 2024-04-28T12:48:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.337+0000 7f082be00640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f08100a3c30 0x7f08100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.338+0000 7f082be00640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f08100a3c30 0x7f08100a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f081c000f80 tx=0x7f081c02f870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:54.337 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.341+0000 7f0829600640 1 -- 172.21.15.106:0/1765135634 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f081409c140 con 0x7f082c076910 2024-04-28T12:48:54.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:54 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-28T12:48:54.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:54 smithi171 ceph-mon[63127]: osdmap e82: 12 total, 12 up, 12 in 2024-04-28T12:48:54.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:54 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1483456306' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-28T12:48:54.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:54 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-28T12:48:54.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:54 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-28T12:48:54.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:54 smithi129 ceph-mon[62418]: osdmap e82: 12 total, 12 up, 12 in 2024-04-28T12:48:54.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:54 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1483456306' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-28T12:48:54.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:54 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-28T12:48:54.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:54 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-28T12:48:54.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:54 smithi106 ceph-mon[28081]: osdmap e82: 12 total, 12 up, 12 in 2024-04-28T12:48:54.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:54 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1483456306' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-28T12:48:54.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:54 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-28T12:48:54.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:54.591 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:54.595+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f082c0635a0 con 0x7f082c076910 2024-04-28T12:48:55.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.106+0000 7f0829600640 1 -- 172.21.15.106:0/1765135634 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f08140a0440 con 0x7f082c076910 2024-04-28T12:48:55.102 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 2 and data pool 3 2024-04-28T12:48:55.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.111+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f08100a3c30 msgr2=0x7f08100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:55.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.111+0000 7f0832a00640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f08100a3c30 0x7f08100a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f081c000f80 tx=0x7f081c02f870 comp rx=0 tx=0).stop 2024-04-28T12:48:55.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.111+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f082c076910 msgr2=0x7f082c19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:55.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.111+0000 7f0832a00640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f082c076910 0x7f082c19b310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f081400d6d0 tx=0x7f081400db90 comp rx=0 tx=0).stop 2024-04-28T12:48:55.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.112+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 shutdown_connections 2024-04-28T12:48:55.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.112+0000 7f0832a00640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f08100a3c30 0x7f08100a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.112+0000 7f0832a00640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f082c10a810 0x7f082c1a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.112+0000 7f0832a00640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f082c076910 0x7f082c19b310 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.112+0000 7f0832a00640 1 --2- 172.21.15.106:0/1765135634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f082c075330 0x7f082c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.112+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 >> 172.21.15.106:0/1765135634 conn(0x7f082c0fdf10 msgr2=0x7f082c1067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:55.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.113+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 shutdown_connections 2024-04-28T12:48:55.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.113+0000 7f0832a00640 1 -- 172.21.15.106:0/1765135634 wait complete. 2024-04-28T12:48:55.130 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-28T12:48:55.130 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-28T12:48:55.320 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.324+0000 7f15b7e00640 1 -- 172.21.15.106:0/2274537711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15b8105b80 msgr2=0x7f15b8106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:55.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.324+0000 7f15b7e00640 1 --2- 172.21.15.106:0/2274537711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15b8105b80 0x7f15b8106000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009be0 tx=0x7f15a402a6d0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.324+0000 7f15b7e00640 1 -- 172.21.15.106:0/2274537711 shutdown_connections 2024-04-28T12:48:55.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.324+0000 7f15b7e00640 1 --2- 172.21.15.106:0/2274537711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15b8106540 0x7f15b810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.324+0000 7f15b7e00640 1 --2- 172.21.15.106:0/2274537711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15b8105b80 0x7f15b8106000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.324+0000 7f15b7e00640 1 --2- 172.21.15.106:0/2274537711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b8104990 0x7f15b8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.324+0000 7f15b7e00640 1 -- 172.21.15.106:0/2274537711 >> 172.21.15.106:0/2274537711 conn(0x7f15b80fff00 msgr2=0x7f15b8102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:55.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.324+0000 7f15b7e00640 1 -- 172.21.15.106:0/2274537711 shutdown_connections 2024-04-28T12:48:55.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.324+0000 7f15b7e00640 1 -- 172.21.15.106:0/2274537711 wait complete. 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b7e00640 1 Processor -- start 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b7e00640 1 -- start start 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15b8104990 0x7f15b819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15b8105b80 0x7f15b819b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b8106540 0x7f15b81a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b7e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f15b81104c0 con 0x7f15b8106540 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b7e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8110340 con 0x7f15b8105b80 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b7e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f15b8110640 con 0x7f15b8104990 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b8106540 0x7f15b81a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b6000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15b8105b80 0x7f15b819b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b6000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15b8105b80 0x7f15b819b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44080/0 (socket says 172.21.15.106:44080) 2024-04-28T12:48:55.323 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b6000640 1 -- 172.21.15.106:0/1402843325 learned_addr learned my addr 172.21.15.106:0/1402843325 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:55.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.326+0000 7f15b6a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15b8104990 0x7f15b819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:55.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.327+0000 7f15b6a00640 1 -- 172.21.15.106:0/1402843325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15b8105b80 msgr2=0x7f15b819b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:55.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.327+0000 7f15b6a00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15b8105b80 0x7f15b819b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.327+0000 7f15b6a00640 1 -- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b8106540 msgr2=0x7f15b81a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:55.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.327+0000 7f15b6a00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b8106540 0x7f15b81a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.327+0000 7f15b6a00640 1 -- 172.21.15.106:0/1402843325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b81a2750 con 0x7f15b8104990 2024-04-28T12:48:55.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.327+0000 7f15b7400640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b8106540 0x7f15b81a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:55.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.327+0000 7f15b6000640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15b8105b80 0x7f15b819b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:55.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.327+0000 7f15b6a00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15b8104990 0x7f15b819ab30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f15a8005fd0 tx=0x7f15a800ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:55.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.328+0000 7f15bca00640 1 -- 172.21.15.106:0/1402843325 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a800ccc0 con 0x7f15b8104990 2024-04-28T12:48:55.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.328+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b81a2a40 con 0x7f15b8104990 2024-04-28T12:48:55.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.328+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b81a2fe0 con 0x7f15b8104990 2024-04-28T12:48:55.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.328+0000 7f15bca00640 1 -- 172.21.15.106:0/1402843325 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15a80044d0 con 0x7f15b8104990 2024-04-28T12:48:55.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.328+0000 7f15bca00640 1 -- 172.21.15.106:0/1402843325 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8003d40 con 0x7f15b8104990 2024-04-28T12:48:55.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.329+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b810bd30 con 0x7f15b8104990 2024-04-28T12:48:55.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.331+0000 7f15bca00640 1 -- 172.21.15.106:0/1402843325 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f15a8004020 con 0x7f15b8104990 2024-04-28T12:48:55.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.332+0000 7f15bca00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f159c0a3c30 0x7f159c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:55.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.333+0000 7f15bca00640 1 -- 172.21.15.106:0/1402843325 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f15a80d0510 con 0x7f15b8104990 2024-04-28T12:48:55.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.333+0000 7f15b6000640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f159c0a3c30 0x7f159c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:55.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.333+0000 7f15b6000640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f159c0a3c30 0x7f159c0a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f15a40099a0 tx=0x7f15a4030970 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:55.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.337+0000 7f15bca00640 1 -- 172.21.15.106:0/1402843325 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f15a809cfa0 con 0x7f15b8104990 2024-04-28T12:48:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: pgmap v311: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-28T12:48:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: osdmap e83: 12 total, 12 up, 12 in 2024-04-28T12:48:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1765135634' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-28T12:48:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-28T12:48:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-28T12:48:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: osdmap e84: 12 total, 12 up, 12 in 2024-04-28T12:48:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:55 smithi171 ceph-mon[63127]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: pgmap v311: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: osdmap e83: 12 total, 12 up, 12 in 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1765135634' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: osdmap e84: 12 total, 12 up, 12 in 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:55.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:55.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:55 smithi129 ceph-mon[62418]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-28T12:48:55.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.550+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f15b802dad0 con 0x7f15b8104990 2024-04-28T12:48:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: pgmap v311: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T12:48:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-28T12:48:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: osdmap e83: 12 total, 12 up, 12 in 2024-04-28T12:48:55.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1765135634' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-28T12:48:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-28T12:48:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-28T12:48:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: osdmap e84: 12 total, 12 up, 12 in 2024-04-28T12:48:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:48:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:48:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:48:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:55 smithi106 ceph-mon[28081]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-28T12:48:55.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.950+0000 7f15bca00640 1 -- 172.21.15.106:0/1402843325 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f15b802dad0 con 0x7f15b8104990 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.954+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f159c0a3c30 msgr2=0x7f159c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.954+0000 7f15b7e00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f159c0a3c30 0x7f159c0a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f15a40099a0 tx=0x7f15a4030970 comp rx=0 tx=0).stop 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.954+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15b8104990 msgr2=0x7f15b819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.954+0000 7f15b7e00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15b8104990 0x7f15b819ab30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f15a8005fd0 tx=0x7f15a800ec20 comp rx=0 tx=0).stop 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.955+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 shutdown_connections 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.955+0000 7f15b7e00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f159c0a3c30 0x7f159c0a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.955+0000 7f15b7e00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f15b8106540 0x7f15b81a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.955+0000 7f15b7e00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f15b8105b80 0x7f15b819b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.955+0000 7f15b7e00640 1 --2- 172.21.15.106:0/1402843325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f15b8104990 0x7f15b819ab30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.955+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 >> 172.21.15.106:0/1402843325 conn(0x7f15b80fff00 msgr2=0x7f15b8108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.955+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 shutdown_connections 2024-04-28T12:48:55.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:55.955+0000 7f15b7e00640 1 -- 172.21.15.106:0/1402843325 wait complete. 2024-04-28T12:48:55.975 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-28T12:48:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.189+0000 7f76db400640 1 -- 172.21.15.106:0/2853764490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4104b90 msgr2=0x7f76d4104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.189+0000 7f76db400640 1 --2- 172.21.15.106:0/2853764490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4104b90 0x7f76d4104fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009900 tx=0x7f76c802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:48:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.189+0000 7f76db400640 1 -- 172.21.15.106:0/2853764490 shutdown_connections 2024-04-28T12:48:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.189+0000 7f76db400640 1 --2- 172.21.15.106:0/2853764490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76d4106740 0x7f76d410cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.189+0000 7f76db400640 1 --2- 172.21.15.106:0/2853764490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4105d80 0x7f76d4106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.189+0000 7f76db400640 1 --2- 172.21.15.106:0/2853764490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4104b90 0x7f76d4104fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:56.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.189+0000 7f76db400640 1 -- 172.21.15.106:0/2853764490 >> 172.21.15.106:0/2853764490 conn(0x7f76d4100120 msgr2=0x7f76d4102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.189+0000 7f76db400640 1 -- 172.21.15.106:0/2853764490 shutdown_connections 2024-04-28T12:48:56.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.189+0000 7f76db400640 1 -- 172.21.15.106:0/2853764490 wait complete. 2024-04-28T12:48:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.190+0000 7f76db400640 1 Processor -- start 2024-04-28T12:48:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.190+0000 7f76db400640 1 -- start start 2024-04-28T12:48:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.190+0000 7f76db400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76d4104b90 0x7f76d419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4105d80 0x7f76d419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76db400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4106740 0x7f76d41a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76db400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4110710 con 0x7f76d4105d80 2024-04-28T12:48:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76db400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4110590 con 0x7f76d4106740 2024-04-28T12:48:56.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76db400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4110890 con 0x7f76d4104b90 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4105d80 0x7f76d419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d8a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76d4104b90 0x7f76d419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4106740 0x7f76d41a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4105d80 0x7f76d419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48204/0 (socket says 172.21.15.106:48204) 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d8a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76d4104b90 0x7f76d419ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:51236/0 (socket says 172.21.15.106:51236) 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d3e00640 1 -- 172.21.15.106:0/3486517309 learned_addr learned my addr 172.21.15.106:0/3486517309 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4106740 0x7f76d41a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44104/0 (socket says 172.21.15.106:44104) 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d9400640 1 -- 172.21.15.106:0/3486517309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76d4104b90 msgr2=0x7f76d419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d9400640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76d4104b90 0x7f76d419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d9400640 1 -- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4105d80 msgr2=0x7f76d419b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d9400640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4105d80 0x7f76d419b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d9400640 1 -- 172.21.15.106:0/3486517309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d41a2a00 con 0x7f76d4106740 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.191+0000 7f76d3e00640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4105d80 0x7f76d419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:48:56.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.192+0000 7f76d9400640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4106740 0x7f76d41a2390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009fd0 tx=0x7f76cc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.192+0000 7f76d1600640 1 -- 172.21.15.106:0/3486517309 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00ce20 con 0x7f76d4106740 2024-04-28T12:48:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.192+0000 7f76d1600640 1 -- 172.21.15.106:0/3486517309 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76cc005020 con 0x7f76d4106740 2024-04-28T12:48:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.192+0000 7f76d1600640 1 -- 172.21.15.106:0/3486517309 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc010690 con 0x7f76d4106740 2024-04-28T12:48:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.192+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d41a2cf0 con 0x7f76d4106740 2024-04-28T12:48:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.192+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a3190 con 0x7f76d4106740 2024-04-28T12:48:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.193+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d410beb0 con 0x7f76d4106740 2024-04-28T12:48:56.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.195+0000 7f76d1600640 1 -- 172.21.15.106:0/3486517309 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f76cc0054d0 con 0x7f76d4106740 2024-04-28T12:48:56.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.196+0000 7f76d1600640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f76bc0a3c30 0x7f76bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:56.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.196+0000 7f76d1600640 1 -- 172.21.15.106:0/3486517309 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f76cc0d0a80 con 0x7f76d4106740 2024-04-28T12:48:56.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.197+0000 7f76d8a00640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f76bc0a3c30 0x7f76bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:56.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.197+0000 7f76d8a00640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f76bc0a3c30 0x7f76bc0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76c8005ab0 tx=0x7f76c8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:56.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.201+0000 7f76d1600640 1 -- 172.21.15.106:0/3486517309 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f76cc0d2e20 con 0x7f76d4106740 2024-04-28T12:48:56.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:56 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-28T12:48:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:56 smithi106 ceph-mon[28081]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-28T12:48:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:56 smithi106 ceph-mon[28081]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-28T12:48:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:56 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1402843325' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-28T12:48:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:56 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-28T12:48:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:56 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-28T12:48:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:56 smithi106 ceph-mon[28081]: osdmap e85: 12 total, 12 up, 12 in 2024-04-28T12:48:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:56 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] up:active 2024-04-28T12:48:56.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:56 smithi106 ceph-mon[28081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-28T12:48:56.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:56.402+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f76d40635d0 con 0x7f76d4106740 2024-04-28T12:48:56.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:56 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-28T12:48:56.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:56 smithi171 ceph-mon[63127]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-28T12:48:56.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:56 smithi171 ceph-mon[63127]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-28T12:48:56.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:56 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1402843325' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-28T12:48:56.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:56 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-28T12:48:56.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:56 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-28T12:48:56.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:56 smithi171 ceph-mon[63127]: osdmap e85: 12 total, 12 up, 12 in 2024-04-28T12:48:56.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:56 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] up:active 2024-04-28T12:48:56.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:56 smithi171 ceph-mon[63127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-28T12:48:56.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:56 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-28T12:48:56.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:56 smithi129 ceph-mon[62418]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-28T12:48:56.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:56 smithi129 ceph-mon[62418]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-28T12:48:56.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:56 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1402843325' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-28T12:48:56.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:56 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-28T12:48:56.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:56 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-28T12:48:56.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:56 smithi129 ceph-mon[62418]: osdmap e85: 12 total, 12 up, 12 in 2024-04-28T12:48:56.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:56 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] up:active 2024-04-28T12:48:56.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:56 smithi129 ceph-mon[62418]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-28T12:48:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:57 smithi171 ceph-mon[63127]: pgmap v315: 97 pgs: 36 active+clean, 19 creating+peering, 42 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-04-28T12:48:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:57 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3486517309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-28T12:48:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:57 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-28T12:48:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:57 smithi171 ceph-mon[63127]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-28T12:48:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:57 smithi171 ceph-mon[63127]: Cluster is now healthy 2024-04-28T12:48:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:57 smithi129 ceph-mon[62418]: pgmap v315: 97 pgs: 36 active+clean, 19 creating+peering, 42 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-04-28T12:48:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:57 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3486517309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-28T12:48:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:57 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-28T12:48:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:57 smithi129 ceph-mon[62418]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-28T12:48:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:57 smithi129 ceph-mon[62418]: Cluster is now healthy 2024-04-28T12:48:57.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:57 smithi106 ceph-mon[28081]: pgmap v315: 97 pgs: 36 active+clean, 19 creating+peering, 42 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-04-28T12:48:57.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:57 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3486517309' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-28T12:48:57.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:57 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-28T12:48:57.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:57 smithi106 ceph-mon[28081]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-28T12:48:57.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:57 smithi106 ceph-mon[28081]: Cluster is now healthy 2024-04-28T12:48:58.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.153+0000 7f76d1600640 1 -- 172.21.15.106:0/3486517309 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f76cc09d3d0 con 0x7f76d4106740 2024-04-28T12:48:58.156 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data_ec' created 2024-04-28T12:48:58.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.162+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f76bc0a3c30 msgr2=0x7f76bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:58.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.163+0000 7f76db400640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f76bc0a3c30 0x7f76bc0a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76c8005ab0 tx=0x7f76c8005a20 comp rx=0 tx=0).stop 2024-04-28T12:48:58.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.163+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4106740 msgr2=0x7f76d41a2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:58.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.163+0000 7f76db400640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4106740 0x7f76d41a2390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009fd0 tx=0x7f76cc00ef10 comp rx=0 tx=0).stop 2024-04-28T12:48:58.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.164+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 shutdown_connections 2024-04-28T12:48:58.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.164+0000 7f76db400640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f76bc0a3c30 0x7f76bc0a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.164+0000 7f76db400640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d4105d80 0x7f76d419b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.164+0000 7f76db400640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76d4106740 0x7f76d41a2390 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.164+0000 7f76db400640 1 --2- 172.21.15.106:0/3486517309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76d4104b90 0x7f76d419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.164+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 >> 172.21.15.106:0/3486517309 conn(0x7f76d4100120 msgr2=0x7f76d4109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:58.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.165+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 shutdown_connections 2024-04-28T12:48:58.162 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.165+0000 7f76db400640 1 -- 172.21.15.106:0/3486517309 wait complete. 2024-04-28T12:48:58.183 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-28T12:48:58.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.372+0000 7f5e46e00640 1 -- 172.21.15.106:0/3191487737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e40109fc0 msgr2=0x7f5e4010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.372+0000 7f5e46e00640 1 --2- 172.21.15.106:0/3191487737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e40109fc0 0x7f5e4010a440 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009900 tx=0x7f5e2c02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.372+0000 7f5e46e00640 1 -- 172.21.15.106:0/3191487737 shutdown_connections 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.372+0000 7f5e46e00640 1 --2- 172.21.15.106:0/3191487737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e4010a980 0x7f5e401112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.372+0000 7f5e46e00640 1 --2- 172.21.15.106:0/3191487737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e40109fc0 0x7f5e4010a440 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.372+0000 7f5e46e00640 1 --2- 172.21.15.106:0/3191487737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e40108e70 0x7f5e40109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.372+0000 7f5e46e00640 1 -- 172.21.15.106:0/3191487737 >> 172.21.15.106:0/3191487737 conn(0x7f5e40077e00 msgr2=0x7f5e4007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.372+0000 7f5e46e00640 1 -- 172.21.15.106:0/3191487737 shutdown_connections 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.373+0000 7f5e46e00640 1 -- 172.21.15.106:0/3191487737 wait complete. 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.373+0000 7f5e46e00640 1 Processor -- start 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.373+0000 7f5e46e00640 1 -- start start 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e46e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e40108e70 0x7f5e4019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e46e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e40109fc0 0x7f5e4019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:58.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e46e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e4010a980 0x7f5e401a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e46e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40114a10 con 0x7f5e40109fc0 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e46e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40114890 con 0x7f5e40108e70 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e46e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40114b90 con 0x7f5e4010a980 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e45000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e40109fc0 0x7f5e4019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e46400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e4010a980 0x7f5e401a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e45000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e40109fc0 0x7f5e4019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48226/0 (socket says 172.21.15.106:48226) 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e45a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e40108e70 0x7f5e4019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.374+0000 7f5e45000640 1 -- 172.21.15.106:0/1664216459 learned_addr learned my addr 172.21.15.106:0/1664216459 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e46400640 1 -- 172.21.15.106:0/1664216459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e40108e70 msgr2=0x7f5e4019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e46400640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e40108e70 0x7f5e4019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e46400640 1 -- 172.21.15.106:0/1664216459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e40109fc0 msgr2=0x7f5e4019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e46400640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e40109fc0 0x7f5e4019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e46400640 1 -- 172.21.15.106:0/1664216459 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e401a6cd0 con 0x7f5e4010a980 2024-04-28T12:48:58.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e46400640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e4010a980 0x7f5e401a6660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009fd0 tx=0x7f5e3800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:58.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e3e000640 1 -- 172.21.15.106:0/1664216459 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800ce20 con 0x7f5e4010a980 2024-04-28T12:48:58.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e3e000640 1 -- 172.21.15.106:0/1664216459 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e38005020 con 0x7f5e4010a980 2024-04-28T12:48:58.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e401a6fc0 con 0x7f5e4010a980 2024-04-28T12:48:58.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e3e000640 1 -- 172.21.15.106:0/1664216459 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38010690 con 0x7f5e4010a980 2024-04-28T12:48:58.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.375+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e401a7430 con 0x7f5e4010a980 2024-04-28T12:48:58.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.377+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e40110180 con 0x7f5e4010a980 2024-04-28T12:48:58.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.379+0000 7f5e3e000640 1 -- 172.21.15.106:0/1664216459 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5e380054d0 con 0x7f5e4010a980 2024-04-28T12:48:58.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.380+0000 7f5e3e000640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5e280a3bb0 0x7f5e280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:58.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.380+0000 7f5e3e000640 1 -- 172.21.15.106:0/1664216459 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7f5e380d0d40 con 0x7f5e4010a980 2024-04-28T12:48:58.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.380+0000 7f5e45a00640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5e280a3bb0 0x7f5e280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:58.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.381+0000 7f5e45a00640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5e280a3bb0 0x7f5e280a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009bd0 tx=0x7f5e34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:58.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.384+0000 7f5e3e000640 1 -- 172.21.15.106:0/1664216459 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5e3809d4a0 con 0x7f5e4010a980 2024-04-28T12:48:58.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:58 smithi171 ceph-mon[63127]: osdmap e86: 12 total, 12 up, 12 in 2024-04-28T12:48:58.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:58 smithi129 ceph-mon[62418]: osdmap e86: 12 total, 12 up, 12 in 2024-04-28T12:48:58.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:58 smithi106 ceph-mon[28081]: osdmap e86: 12 total, 12 up, 12 in 2024-04-28T12:48:58.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:58.585+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f5e40063530 con 0x7f5e4010a980 2024-04-28T12:48:59.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.153+0000 7f5e3e000640 1 -- 172.21.15.106:0/1664216459 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7f5e3809cda0 con 0x7f5e4010a980 2024-04-28T12:48:59.150 INFO:teuthology.orchestra.run.smithi106.stderr:set pool 4 allow_ec_overwrites to true 2024-04-28T12:48:59.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.158+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5e280a3bb0 msgr2=0x7f5e280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:59.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.158+0000 7f5e46e00640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5e280a3bb0 0x7f5e280a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009bd0 tx=0x7f5e34005f90 comp rx=0 tx=0).stop 2024-04-28T12:48:59.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.159+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e4010a980 msgr2=0x7f5e401a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:59.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.159+0000 7f5e46e00640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e4010a980 0x7f5e401a6660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009fd0 tx=0x7f5e3800ef10 comp rx=0 tx=0).stop 2024-04-28T12:48:59.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.159+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 shutdown_connections 2024-04-28T12:48:59.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.159+0000 7f5e46e00640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5e280a3bb0 0x7f5e280a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.159+0000 7f5e46e00640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e40109fc0 0x7f5e4019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.159+0000 7f5e46e00640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e4010a980 0x7f5e401a6660 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.159+0000 7f5e46e00640 1 --2- 172.21.15.106:0/1664216459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5e40108e70 0x7f5e4019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.159+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 >> 172.21.15.106:0/1664216459 conn(0x7f5e40077e00 msgr2=0x7f5e4010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:59.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.162+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 shutdown_connections 2024-04-28T12:48:59.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.162+0000 7f5e46e00640 1 -- 172.21.15.106:0/1664216459 wait complete. 2024-04-28T12:48:59.195 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-28T12:48:59.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.373+0000 7f9a8f800640 1 -- 172.21.15.106:0/2966765977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88108660 msgr2=0x7f9a8810aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:59.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.373+0000 7f9a8f800640 1 --2- 172.21.15.106:0/2966765977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88108660 0x7f9a8810aab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009900 tx=0x7f9a7802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.373+0000 7f9a8f800640 1 -- 172.21.15.106:0/2966765977 shutdown_connections 2024-04-28T12:48:59.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.373+0000 7f9a8f800640 1 --2- 172.21.15.106:0/2966765977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88108660 0x7f9a8810aab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.373+0000 7f9a8f800640 1 --2- 172.21.15.106:0/2966765977 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a88105d30 0x7f9a88108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.373+0000 7f9a8f800640 1 --2- 172.21.15.106:0/2966765977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a88103400 0x7f9a881057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.373+0000 7f9a8f800640 1 -- 172.21.15.106:0/2966765977 >> 172.21.15.106:0/2966765977 conn(0x7f9a880fcdf0 msgr2=0x7f9a880ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:48:59.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.373+0000 7f9a8f800640 1 -- 172.21.15.106:0/2966765977 shutdown_connections 2024-04-28T12:48:59.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.373+0000 7f9a8f800640 1 -- 172.21.15.106:0/2966765977 wait complete. 2024-04-28T12:48:59.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.374+0000 7f9a8f800640 1 Processor -- start 2024-04-28T12:48:59.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8f800640 1 -- start start 2024-04-28T12:48:59.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8f800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88103400 0x7f9a8819adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:59.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8f800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a88105d30 0x7f9a8819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:59.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8f800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a88108660 0x7f9a881a2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:59.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8f800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8810e520 con 0x7f9a88108660 2024-04-28T12:48:59.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8f800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8810e3a0 con 0x7f9a88103400 2024-04-28T12:48:59.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8f800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8810e6a0 con 0x7f9a88105d30 2024-04-28T12:48:59.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88103400 0x7f9a8819adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a88108660 0x7f9a881a2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88103400 0x7f9a8819adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:58324/0 (socket says 172.21.15.106:58324) 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8ce00640 1 -- 172.21.15.106:0/751346608 learned_addr learned my addr 172.21.15.106:0/751346608 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a87e00640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a88105d30 0x7f9a8819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8ce00640 1 -- 172.21.15.106:0/751346608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a88105d30 msgr2=0x7f9a8819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.375+0000 7f9a8ce00640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a88105d30 0x7f9a8819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.376+0000 7f9a8ce00640 1 -- 172.21.15.106:0/751346608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a88108660 msgr2=0x7f9a881a2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.376+0000 7f9a8ce00640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a88108660 0x7f9a881a2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.376+0000 7f9a8ce00640 1 -- 172.21.15.106:0/751346608 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a881a29d0 con 0x7f9a88103400 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.376+0000 7f9a8ce00640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88103400 0x7f9a8819adb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000ebc0 tx=0x7f9a8000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.376+0000 7f9a85600640 1 -- 172.21.15.106:0/751346608 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80010630 con 0x7f9a88103400 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.376+0000 7f9a85600640 1 -- 172.21.15.106:0/751346608 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a80010c80 con 0x7f9a88103400 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.376+0000 7f9a85600640 1 -- 172.21.15.106:0/751346608 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80018b60 con 0x7f9a88103400 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.376+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a881a2cc0 con 0x7f9a88103400 2024-04-28T12:48:59.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.376+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a881a31c0 con 0x7f9a88103400 2024-04-28T12:48:59.374 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.377+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8810a150 con 0x7f9a88103400 2024-04-28T12:48:59.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.379+0000 7f9a85600640 1 -- 172.21.15.106:0/751346608 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9a80018d00 con 0x7f9a88103400 2024-04-28T12:48:59.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.381+0000 7f9a85600640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9a6c0a3bb0 0x7f9a6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:48:59.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.381+0000 7f9a85600640 1 -- 172.21.15.106:0/751346608 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f9a800d0c50 con 0x7f9a88103400 2024-04-28T12:48:59.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.381+0000 7f9a87e00640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9a6c0a3bb0 0x7f9a6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:48:59.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.381+0000 7f9a87e00640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9a6c0a3bb0 0x7f9a6c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a8819c2c0 tx=0x7f9a70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:48:59.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.385+0000 7f9a85600640 1 -- 172.21.15.106:0/751346608 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9a8009cd60 con 0x7f9a88103400 2024-04-28T12:48:59.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:59 smithi171 ceph-mon[63127]: pgmap v317: 97 pgs: 62 active+clean, 19 creating+peering, 16 unknown; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-04-28T12:48:59.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:59 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-28T12:48:59.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:59 smithi171 ceph-mon[63127]: osdmap e87: 12 total, 12 up, 12 in 2024-04-28T12:48:59.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:59 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1664216459' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-28T12:48:59.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:48:59 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-28T12:48:59.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:59 smithi129 ceph-mon[62418]: pgmap v317: 97 pgs: 62 active+clean, 19 creating+peering, 16 unknown; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-04-28T12:48:59.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:59 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-28T12:48:59.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:59 smithi129 ceph-mon[62418]: osdmap e87: 12 total, 12 up, 12 in 2024-04-28T12:48:59.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:59 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1664216459' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-28T12:48:59.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:48:59 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-28T12:48:59.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:59 smithi106 ceph-mon[28081]: pgmap v317: 97 pgs: 62 active+clean, 19 creating+peering, 16 unknown; 578 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 2 op/s 2024-04-28T12:48:59.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:59 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-28T12:48:59.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:59 smithi106 ceph-mon[28081]: osdmap e87: 12 total, 12 up, 12 in 2024-04-28T12:48:59.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:59 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1664216459' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-28T12:48:59.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:48:59 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-28T12:48:59.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:48:59.627+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f9a880635d0 con 0x7f9a88103400 2024-04-28T12:49:00.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.185+0000 7f9a85600640 1 -- 172.21.15.106:0/751346608 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f9a800a1060 con 0x7f9a88103400 2024-04-28T12:49:00.181 INFO:teuthology.orchestra.run.smithi106.stderr:added data pool 4 to fsmap 2024-04-28T12:49:00.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9a6c0a3bb0 msgr2=0x7f9a6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9a6c0a3bb0 0x7f9a6c0a6070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9a8819c2c0 tx=0x7f9a70005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:00.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88103400 msgr2=0x7f9a8819adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88103400 0x7f9a8819adb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000ebc0 tx=0x7f9a8000c6a0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 shutdown_connections 2024-04-28T12:49:00.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9a6c0a3bb0 0x7f9a6c0a6070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a88108660 0x7f9a881a2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a88105d30 0x7f9a8819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 --2- 172.21.15.106:0/751346608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a88103400 0x7f9a8819adb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 >> 172.21.15.106:0/751346608 conn(0x7f9a880fcdf0 msgr2=0x7f9a88102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:00.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 shutdown_connections 2024-04-28T12:49:00.187 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.190+0000 7f9a8f800640 1 -- 172.21.15.106:0/751346608 wait complete. 2024-04-28T12:49:00.206 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:49:00.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.406+0000 7fe7bd800640 1 -- 172.21.15.106:0/2335395713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8109f80 msgr2=0x7fe7b810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.406+0000 7fe7bd800640 1 --2- 172.21.15.106:0/2335395713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8109f80 0x7fe7b810a400 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009900 tx=0x7fe7a002a410 comp rx=0 tx=0).stop 2024-04-28T12:49:00.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.406+0000 7fe7bd800640 1 -- 172.21.15.106:0/2335395713 shutdown_connections 2024-04-28T12:49:00.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.406+0000 7fe7bd800640 1 --2- 172.21.15.106:0/2335395713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8109f80 0x7fe7b810a400 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.406+0000 7fe7bd800640 1 --2- 172.21.15.106:0/2335395713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7b810a940 0x7fe7b81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.406+0000 7fe7bd800640 1 --2- 172.21.15.106:0/2335395713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8108d90 0x7fe7b81091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.406+0000 7fe7bd800640 1 -- 172.21.15.106:0/2335395713 >> 172.21.15.106:0/2335395713 conn(0x7fe7b8077d30 msgr2=0x7fe7b807a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:00.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.406+0000 7fe7bd800640 1 -- 172.21.15.106:0/2335395713 shutdown_connections 2024-04-28T12:49:00.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.406+0000 7fe7bd800640 1 -- 172.21.15.106:0/2335395713 wait complete. 2024-04-28T12:49:00.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.407+0000 7fe7bd800640 1 Processor -- start 2024-04-28T12:49:00.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.407+0000 7fe7bd800640 1 -- start start 2024-04-28T12:49:00.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.407+0000 7fe7bd800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8108d90 0x7fe7b819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:00.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7bd800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8109f80 0x7fe7b819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7bd800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7b810a940 0x7fe7b81a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7bd800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8114910 con 0x7fe7b8108d90 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7bd800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8114790 con 0x7fe7b8109f80 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7bd800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8114a90 con 0x7fe7b810a940 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7b7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8108d90 0x7fe7b819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7b7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8108d90 0x7fe7b819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40772/0 (socket says 172.21.15.106:40772) 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7b7e00640 1 -- 172.21.15.106:0/3951766159 learned_addr learned my addr 172.21.15.106:0/3951766159 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7bce00640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7b810a940 0x7fe7b81a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7af400640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8109f80 0x7fe7b819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7b7e00640 1 -- 172.21.15.106:0/3951766159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7b810a940 msgr2=0x7fe7b81a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7b7e00640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7b810a940 0x7fe7b81a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7b7e00640 1 -- 172.21.15.106:0/3951766159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8109f80 msgr2=0x7fe7b819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7b7e00640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8109f80 0x7fe7b819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7b7e00640 1 -- 172.21.15.106:0/3951766159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b81a6bb0 con 0x7fe7b8108d90 2024-04-28T12:49:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.408+0000 7fe7af400640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8109f80 0x7fe7b819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:49:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.409+0000 7fe7b7e00640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8108d90 0x7fe7b819ef90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800e9c0 tx=0x7fe7a800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.409+0000 7fe7b5600640 1 -- 172.21.15.106:0/3951766159 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800cd80 con 0x7fe7b8108d90 2024-04-28T12:49:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.409+0000 7fe7b5600640 1 -- 172.21.15.106:0/3951766159 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a80035f0 con 0x7fe7b8108d90 2024-04-28T12:49:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.409+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b81a6ea0 con 0x7fe7b8108d90 2024-04-28T12:49:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.409+0000 7fe7b5600640 1 -- 172.21.15.106:0/3951766159 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8010690 con 0x7fe7b8108d90 2024-04-28T12:49:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.409+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b81a7300 con 0x7fe7b8108d90 2024-04-28T12:49:00.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.410+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b81100b0 con 0x7fe7b8108d90 2024-04-28T12:49:00.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.412+0000 7fe7b5600640 1 -- 172.21.15.106:0/3951766159 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe7a80037a0 con 0x7fe7b8108d90 2024-04-28T12:49:00.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.414+0000 7fe7b5600640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe79c0a3bb0 0x7fe79c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.414+0000 7fe7b5600640 1 -- 172.21.15.106:0/3951766159 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe7a80d0b60 con 0x7fe7b8108d90 2024-04-28T12:49:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.414+0000 7fe7af400640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe79c0a3bb0 0x7fe79c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:00.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.415+0000 7fe7af400640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe79c0a3bb0 0x7fe79c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0000c00 tx=0x7fe7a00314c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:00.412 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.416+0000 7fe7b5600640 1 -- 172.21.15.106:0/3951766159 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe7a809d2a0 con 0x7fe7b8108d90 2024-04-28T12:49:00.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:00 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-28T12:49:00.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:00 smithi171 ceph-mon[63127]: osdmap e88: 12 total, 12 up, 12 in 2024-04-28T12:49:00.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:00 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/751346608' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-28T12:49:00.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:00 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-28T12:49:00.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:00 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-28T12:49:00.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:00 smithi129 ceph-mon[62418]: osdmap e88: 12 total, 12 up, 12 in 2024-04-28T12:49:00.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:00 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/751346608' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-28T12:49:00.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:00 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-28T12:49:00.513 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:49:00 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: 2024-04-28T12:49:00.074+0000 7f952dbf8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-28T12:49:00.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:00 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-28T12:49:00.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:00 smithi106 ceph-mon[28081]: osdmap e88: 12 total, 12 up, 12 in 2024-04-28T12:49:00.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:00 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/751346608' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-28T12:49:00.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:00 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-28T12:49:00.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.643+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7b802dd80 con 0x7fe7b8108d90 2024-04-28T12:49:00.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.644+0000 7fe7b5600640 1 -- 172.21.15.106:0/3951766159 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11818 (secure 0 0 0) 0x7fe7a809cba0 con 0x7fe7b8108d90 2024-04-28T12:49:00.640 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:00.640 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:48:59.631117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24578,"qdb_cluster":[24578]},"id":1}]} 2024-04-28T12:49:00.641 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-28T12:49:00.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe79c0a3bb0 msgr2=0x7fe79c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe79c0a3bb0 0x7fe79c0a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0000c00 tx=0x7fe7a00314c0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8108d90 msgr2=0x7fe7b819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8108d90 0x7fe7b819ef90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800e9c0 tx=0x7fe7a800ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:00.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 shutdown_connections 2024-04-28T12:49:00.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe79c0a3bb0 0x7fe79c0a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7b810a940 0x7fe7b81a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8109f80 0x7fe7b819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 --2- 172.21.15.106:0/3951766159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7b8108d90 0x7fe7b819ef90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 >> 172.21.15.106:0/3951766159 conn(0x7fe7b8077d30 msgr2=0x7fe7b810d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:00.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 shutdown_connections 2024-04-28T12:49:00.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.647+0000 7fe7bd800640 1 -- 172.21.15.106:0/3951766159 wait complete. 2024-04-28T12:49:00.660 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:49:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.865+0000 7f1fc1c00640 1 -- 172.21.15.106:0/435182487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fbc104530 msgr2=0x7f1fbc10adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.865+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/435182487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fbc104530 0x7f1fbc10adc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009860 tx=0x7f1fb402a340 comp rx=0 tx=0).stop 2024-04-28T12:49:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.866+0000 7f1fc1c00640 1 -- 172.21.15.106:0/435182487 shutdown_connections 2024-04-28T12:49:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.866+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/435182487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fbc103b70 0x7f1fbc103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.866+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/435182487 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fbc104530 0x7f1fbc10adc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.866+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/435182487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fbc102980 0x7f1fbc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.866+0000 7f1fc1c00640 1 -- 172.21.15.106:0/435182487 >> 172.21.15.106:0/435182487 conn(0x7f1fbc0fdf10 msgr2=0x7f1fbc100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.866+0000 7f1fc1c00640 1 -- 172.21.15.106:0/435182487 shutdown_connections 2024-04-28T12:49:00.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.866+0000 7f1fc1c00640 1 -- 172.21.15.106:0/435182487 wait complete. 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.867+0000 7f1fc1c00640 1 Processor -- start 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.867+0000 7f1fc1c00640 1 -- start start 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.867+0000 7f1fc1c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fbc102980 0x7f1fbc0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.867+0000 7f1fc1c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fbc103b70 0x7f1fbc074d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.867+0000 7f1fc1c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fbc104530 0x7f1fbc075260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.867+0000 7f1fc1c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc10e4b0 con 0x7f1fbc103b70 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.867+0000 7f1fc1c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc10e330 con 0x7f1fbc104530 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.867+0000 7f1fc1c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc10e630 con 0x7f1fbc102980 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.867+0000 7f1fba000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fbc103b70 0x7f1fbc074d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fba000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fbc103b70 0x7f1fbc074d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40800/0 (socket says 172.21.15.106:40800) 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fba000640 1 -- 172.21.15.106:0/3800270662 learned_addr learned my addr 172.21.15.106:0/3800270662 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:00.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fbaa00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fbc102980 0x7f1fbc0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fbb400640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fbc104530 0x7f1fbc075260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fba000640 1 -- 172.21.15.106:0/3800270662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fbc102980 msgr2=0x7f1fbc0766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fba000640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fbc102980 0x7f1fbc0766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fba000640 1 -- 172.21.15.106:0/3800270662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fbc104530 msgr2=0x7f1fbc075260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fba000640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fbc104530 0x7f1fbc075260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fba000640 1 -- 172.21.15.106:0/3800270662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc075af0 con 0x7f1fbc103b70 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fbaa00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fbc102980 0x7f1fbc0766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fbb400640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fbc104530 0x7f1fbc075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fba000640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fbc103b70 0x7f1fbc074d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400e9c0 tx=0x7f1fa400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fb3400640 1 -- 172.21.15.106:0/3800270662 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa400ccc0 con 0x7f1fbc103b70 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc1a2ca0 con 0x7f1fbc103b70 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.868+0000 7f1fb3400640 1 -- 172.21.15.106:0/3800270662 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fa40054d0 con 0x7f1fbc103b70 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.869+0000 7f1fb3400640 1 -- 172.21.15.106:0/3800270662 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4010430 con 0x7f1fbc103b70 2024-04-28T12:49:00.866 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.869+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc1a31a0 con 0x7f1fbc103b70 2024-04-28T12:49:00.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.870+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fbc109dc0 con 0x7f1fbc103b70 2024-04-28T12:49:00.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.872+0000 7f1fb3400640 1 -- 172.21.15.106:0/3800270662 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1fa4004f80 con 0x7f1fbc103b70 2024-04-28T12:49:00.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.874+0000 7f1fb3400640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1fa00a3bb0 0x7f1fa00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:00.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.874+0000 7f1fb3400640 1 -- 172.21.15.106:0/3800270662 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1fa40d0b90 con 0x7f1fbc103b70 2024-04-28T12:49:00.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.874+0000 7f1fbaa00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1fa00a3bb0 0x7f1fa00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:00.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.875+0000 7f1fbaa00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1fa00a3bb0 0x7f1fa00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc1039d0 tx=0x7f1fac009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:00.874 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:00.878+0000 7f1fb3400640 1 -- 172.21.15.106:0/3800270662 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1fa409d2d0 con 0x7f1fbc103b70 2024-04-28T12:49:01.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.080+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fbc063530 con 0x7f1fbc103b70 2024-04-28T12:49:01.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.082+0000 7f1fb3400640 1 -- 172.21.15.106:0/3800270662 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7f1fa409cbd0 con 0x7f1fbc103b70 2024-04-28T12:49:01.079 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:01.079 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:01.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.085+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1fa00a3bb0 msgr2=0x7f1fa00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.085+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1fa00a3bb0 0x7f1fa00a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc1039d0 tx=0x7f1fac009500 comp rx=0 tx=0).stop 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.085+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fbc103b70 msgr2=0x7f1fbc074d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.085+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fbc103b70 0x7f1fbc074d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1fa400e9c0 tx=0x7f1fa400ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.086+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 shutdown_connections 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.086+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1fa00a3bb0 0x7f1fa00a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.086+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fbc103b70 0x7f1fbc074d20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.086+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fbc104530 0x7f1fbc075260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.086+0000 7f1fc1c00640 1 --2- 172.21.15.106:0/3800270662 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fbc102980 0x7f1fbc0766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.086+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 >> 172.21.15.106:0/3800270662 conn(0x7f1fbc0fdf10 msgr2=0x7f1fbc106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.086+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 shutdown_connections 2024-04-28T12:49:01.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.086+0000 7f1fc1c00640 1 -- 172.21.15.106:0/3800270662 wait complete. 2024-04-28T12:49:01.102 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:49:01.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.308+0000 7efd93600640 1 -- 172.21.15.106:0/2940187744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd8c108f90 msgr2=0x7efd8c1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.308+0000 7efd93600640 1 --2- 172.21.15.106:0/2940187744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd8c108f90 0x7efd8c1093b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009ca0 tx=0x7efd7c02a8c0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.308+0000 7efd93600640 1 -- 172.21.15.106:0/2940187744 shutdown_connections 2024-04-28T12:49:01.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.308+0000 7efd93600640 1 --2- 172.21.15.106:0/2940187744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd8c10ab40 0x7efd8c1113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.308+0000 7efd93600640 1 --2- 172.21.15.106:0/2940187744 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8c10a180 0x7efd8c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.308+0000 7efd93600640 1 --2- 172.21.15.106:0/2940187744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd8c108f90 0x7efd8c1093b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.308+0000 7efd93600640 1 -- 172.21.15.106:0/2940187744 >> 172.21.15.106:0/2940187744 conn(0x7efd8c077f40 msgr2=0x7efd8c07a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:01.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.308+0000 7efd93600640 1 -- 172.21.15.106:0/2940187744 shutdown_connections 2024-04-28T12:49:01.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.308+0000 7efd93600640 1 -- 172.21.15.106:0/2940187744 wait complete. 2024-04-28T12:49:01.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.309+0000 7efd93600640 1 Processor -- start 2024-04-28T12:49:01.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.309+0000 7efd93600640 1 -- start start 2024-04-28T12:49:01.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd93600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8c108f90 0x7efd8c19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:01.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd93600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd8c10a180 0x7efd8c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:01.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd93600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd8c10ab40 0x7efd8c1a6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:01.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd93600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c114ae0 con 0x7efd8c10ab40 2024-04-28T12:49:01.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd93600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c114960 con 0x7efd8c108f90 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd93600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c114c60 con 0x7efd8c10a180 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd90c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8c108f90 0x7efd8c19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd90c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8c108f90 0x7efd8c19f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:58362/0 (socket says 172.21.15.106:58362) 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd90c00640 1 -- 172.21.15.106:0/611878315 learned_addr learned my addr 172.21.15.106:0/611878315 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd91600640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd8c10ab40 0x7efd8c1a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd8be00640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd8c10a180 0x7efd8c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd90c00640 1 -- 172.21.15.106:0/611878315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd8c10a180 msgr2=0x7efd8c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd90c00640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd8c10a180 0x7efd8c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd90c00640 1 -- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd8c10ab40 msgr2=0x7efd8c1a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd90c00640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd8c10ab40 0x7efd8c1a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd90c00640 1 -- 172.21.15.106:0/611878315 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c1a6dd0 con 0x7efd8c108f90 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.310+0000 7efd91600640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd8c10ab40 0x7efd8c1a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.311+0000 7efd90c00640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8c108f90 0x7efd8c19f1b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd7c000f80 tx=0x7efd7c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.311+0000 7efd89600640 1 -- 172.21.15.106:0/611878315 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c02f380 con 0x7efd8c108f90 2024-04-28T12:49:01.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.311+0000 7efd89600640 1 -- 172.21.15.106:0/611878315 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd7c0317c0 con 0x7efd8c108f90 2024-04-28T12:49:01.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.311+0000 7efd89600640 1 -- 172.21.15.106:0/611878315 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c02fc50 con 0x7efd8c108f90 2024-04-28T12:49:01.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.311+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c1a7060 con 0x7efd8c108f90 2024-04-28T12:49:01.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.311+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c1a7560 con 0x7efd8c108f90 2024-04-28T12:49:01.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.312+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8c1102b0 con 0x7efd8c108f90 2024-04-28T12:49:01.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.314+0000 7efd89600640 1 -- 172.21.15.106:0/611878315 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7efd7c004040 con 0x7efd8c108f90 2024-04-28T12:49:01.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.315+0000 7efd89600640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd700a3bb0 0x7efd700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:01.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.315+0000 7efd89600640 1 -- 172.21.15.106:0/611878315 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7efd7c0f1b30 con 0x7efd8c108f90 2024-04-28T12:49:01.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.315+0000 7efd8be00640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd700a3bb0 0x7efd700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:01.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.316+0000 7efd8be00640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd700a3bb0 0x7efd700a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd8c1a06c0 tx=0x7efd74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:01.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.320+0000 7efd89600640 1 -- 172.21.15.106:0/611878315 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efd7c0be270 con 0x7efd8c108f90 2024-04-28T12:49:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:01 smithi171 ceph-mon[63127]: pgmap v320: 129 pgs: 99 active+clean, 18 creating+peering, 12 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-28T12:49:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:01 smithi171 ceph-mon[63127]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-28T12:49:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:01 smithi171 ceph-mon[63127]: osdmap e89: 12 total, 12 up, 12 in 2024-04-28T12:49:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:01 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-28T12:49:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:01 smithi171 ceph-mon[63127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-28T12:49:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:01 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3951766159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:49:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:01 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3800270662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:01 smithi129 ceph-mon[62418]: pgmap v320: 129 pgs: 99 active+clean, 18 creating+peering, 12 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-28T12:49:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:01 smithi129 ceph-mon[62418]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-28T12:49:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:01 smithi129 ceph-mon[62418]: osdmap e89: 12 total, 12 up, 12 in 2024-04-28T12:49:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:01 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-28T12:49:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:01 smithi129 ceph-mon[62418]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-28T12:49:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:01 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3951766159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:49:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:01 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3800270662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:01.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.523+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd8c063530 con 0x7efd8c108f90 2024-04-28T12:49:01.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.525+0000 7efd89600640 1 -- 172.21.15.106:0/611878315 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7efd7c0bdb70 con 0x7efd8c108f90 2024-04-28T12:49:01.522 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:01.522 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:01.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd700a3bb0 msgr2=0x7efd700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd700a3bb0 0x7efd700a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd8c1a06c0 tx=0x7efd74005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:01.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8c108f90 msgr2=0x7efd8c19f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8c108f90 0x7efd8c19f1b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efd7c000f80 tx=0x7efd7c003ba0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 shutdown_connections 2024-04-28T12:49:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd700a3bb0 0x7efd700a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd8c10ab40 0x7efd8c1a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd8c10a180 0x7efd8c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 --2- 172.21.15.106:0/611878315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd8c108f90 0x7efd8c19f1b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 >> 172.21.15.106:0/611878315 conn(0x7efd8c077f40 msgr2=0x7efd8c10d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 shutdown_connections 2024-04-28T12:49:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.528+0000 7efd93600640 1 -- 172.21.15.106:0/611878315 wait complete. 2024-04-28T12:49:01.540 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:01 smithi106 ceph-mon[28081]: pgmap v320: 129 pgs: 99 active+clean, 18 creating+peering, 12 unknown; 579 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-28T12:49:01.540 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:01 smithi106 ceph-mon[28081]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-28T12:49:01.540 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:01 smithi106 ceph-mon[28081]: osdmap e89: 12 total, 12 up, 12 in 2024-04-28T12:49:01.540 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:01 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-28T12:49:01.541 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:01 smithi106 ceph-mon[28081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-28T12:49:01.541 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:01 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3951766159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:49:01.541 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:01 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3800270662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:01.559 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:49:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.766+0000 7fab7b200640 1 -- 172.21.15.106:0/719813433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 msgr2=0x7fab7c10a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.766+0000 7fab7b200640 1 --2- 172.21.15.106:0/719813433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 0x7fab7c10a900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fab74009900 tx=0x7fab7402a410 comp rx=0 tx=0).stop 2024-04-28T12:49:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.767+0000 7fab7b200640 1 -- 172.21.15.106:0/719813433 shutdown_connections 2024-04-28T12:49:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.767+0000 7fab7b200640 1 --2- 172.21.15.106:0/719813433 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 0x7fab7c10a900 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.767+0000 7fab7b200640 1 --2- 172.21.15.106:0/719813433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7c105be0 0x7fab7c107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.767+0000 7fab7b200640 1 --2- 172.21.15.106:0/719813433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab7c1032b0 0x7fab7c1056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.767+0000 7fab7b200640 1 -- 172.21.15.106:0/719813433 >> 172.21.15.106:0/719813433 conn(0x7fab7c0fcbc0 msgr2=0x7fab7c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:01.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.767+0000 7fab7b200640 1 -- 172.21.15.106:0/719813433 shutdown_connections 2024-04-28T12:49:01.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.767+0000 7fab7b200640 1 -- 172.21.15.106:0/719813433 wait complete. 2024-04-28T12:49:01.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7b200640 1 Processor -- start 2024-04-28T12:49:01.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7b200640 1 -- start start 2024-04-28T12:49:01.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab7c1032b0 0x7fab7c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:01.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7c105be0 0x7fab7c19b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:01.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 0x7fab7c1a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:01.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10e300 con 0x7fab7c108510 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10e180 con 0x7fab7c105be0 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c10e480 con 0x7fab7c1032b0 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 0x7fab7c1a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab79e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab7c1032b0 0x7fab7c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.768+0000 7fab7a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 0x7fab7c1a2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40832/0 (socket says 172.21.15.106:40832) 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.769+0000 7fab7a800640 1 -- 172.21.15.106:0/3938559348 learned_addr learned my addr 172.21.15.106:0/3938559348 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.769+0000 7fab79400640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7c105be0 0x7fab7c19b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.769+0000 7fab7a800640 1 -- 172.21.15.106:0/3938559348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab7c1032b0 msgr2=0x7fab7c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.769+0000 7fab7a800640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab7c1032b0 0x7fab7c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.769+0000 7fab7a800640 1 -- 172.21.15.106:0/3938559348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7c105be0 msgr2=0x7fab7c19b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.769+0000 7fab7a800640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7c105be0 0x7fab7c19b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.769+0000 7fab7a800640 1 -- 172.21.15.106:0/3938559348 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c1a2790 con 0x7fab7c108510 2024-04-28T12:49:01.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.769+0000 7fab7a800640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 0x7fab7c1a2120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fab74005ab0 tx=0x7fab74003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.769+0000 7fab72a00640 1 -- 172.21.15.106:0/3938559348 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7403f070 con 0x7fab7c108510 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.770+0000 7fab72a00640 1 -- 172.21.15.106:0/3938559348 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab74031c50 con 0x7fab7c108510 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.770+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c1a2a20 con 0x7fab7c108510 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.770+0000 7fab72a00640 1 -- 172.21.15.106:0/3938559348 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74035030 con 0x7fab7c108510 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.770+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1a2f10 con 0x7fab7c108510 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.771+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7c10a000 con 0x7fab7c108510 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.773+0000 7fab72a00640 1 -- 172.21.15.106:0/3938559348 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fab74042070 con 0x7fab7c108510 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.774+0000 7fab72a00640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab600a3bb0 0x7fab600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.774+0000 7fab72a00640 1 -- 172.21.15.106:0/3938559348 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fab740f1560 con 0x7fab7c108510 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.775+0000 7fab79e00640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab600a3bb0 0x7fab600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.775+0000 7fab79e00640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab600a3bb0 0x7fab600a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab6c005fd0 tx=0x7fab6c009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:01.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:01.777+0000 7fab72a00640 1 -- 172.21.15.106:0/3938559348 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fab740bd650 con 0x7fab7c108510 2024-04-28T12:49:02.011 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.015+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab7c02dad0 con 0x7fab7c108510 2024-04-28T12:49:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.017+0000 7fab72a00640 1 -- 172.21.15.106:0/3938559348 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11818 (secure 0 0 0) 0x7fab740c1950 con 0x7fab7c108510 2024-04-28T12:49:02.013 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:02.013 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24469,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24508,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24520,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24548,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:48:59.631117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24578},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24578,"qdb_cluster":[24578]},"id":1}]} 2024-04-28T12:49:02.013 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 16 2024-04-28T12:49:02.015 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.019+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab600a3bb0 msgr2=0x7fab600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.019+0000 7fab7b200640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab600a3bb0 0x7fab600a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fab6c005fd0 tx=0x7fab6c009760 comp rx=0 tx=0).stop 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.019+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 msgr2=0x7fab7c1a2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.019+0000 7fab7b200640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 0x7fab7c1a2120 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fab74005ab0 tx=0x7fab74003b90 comp rx=0 tx=0).stop 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.020+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 shutdown_connections 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.020+0000 7fab7b200640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab600a3bb0 0x7fab600a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.020+0000 7fab7b200640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab7c108510 0x7fab7c1a2120 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.020+0000 7fab7b200640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab7c105be0 0x7fab7c19b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.020+0000 7fab7b200640 1 --2- 172.21.15.106:0/3938559348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab7c1032b0 0x7fab7c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.020+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 >> 172.21.15.106:0/3938559348 conn(0x7fab7c0fcbc0 msgr2=0x7fab7c102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.020+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 shutdown_connections 2024-04-28T12:49:02.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.020+0000 7fab7b200640 1 -- 172.21.15.106:0/3938559348 wait complete. 2024-04-28T12:49:02.034 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:49:02.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.239+0000 7fcfab000640 1 -- 172.21.15.106:0/998288163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfa4109f80 msgr2=0x7fcfa410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.239+0000 7fcfab000640 1 --2- 172.21.15.106:0/998288163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfa4109f80 0x7fcfa410a400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009900 tx=0x7fcf9002a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.240+0000 7fcfab000640 1 -- 172.21.15.106:0/998288163 shutdown_connections 2024-04-28T12:49:02.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.240+0000 7fcfab000640 1 --2- 172.21.15.106:0/998288163 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfa410a940 0x7fcfa41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.240+0000 7fcfab000640 1 --2- 172.21.15.106:0/998288163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfa4109f80 0x7fcfa410a400 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.240+0000 7fcfab000640 1 --2- 172.21.15.106:0/998288163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfa4108d90 0x7fcfa41091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.240+0000 7fcfab000640 1 -- 172.21.15.106:0/998288163 >> 172.21.15.106:0/998288163 conn(0x7fcfa4077d30 msgr2=0x7fcfa407a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.240+0000 7fcfab000640 1 -- 172.21.15.106:0/998288163 shutdown_connections 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.240+0000 7fcfab000640 1 -- 172.21.15.106:0/998288163 wait complete. 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.240+0000 7fcfab000640 1 Processor -- start 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfab000640 1 -- start start 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfab000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfa4108d90 0x7fcfa419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfab000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfa4109f80 0x7fcfa419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfab000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfa410a940 0x7fcfa41a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfab000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa4114910 con 0x7fcfa410a940 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfab000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa4114790 con 0x7fcfa4108d90 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfab000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa4114a90 con 0x7fcfa4109f80 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfa9c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfa4108d90 0x7fcfa419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfaa600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfa410a940 0x7fcfa41a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfa9c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfa4108d90 0x7fcfa419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:58394/0 (socket says 172.21.15.106:58394) 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfa9c00640 1 -- 172.21.15.106:0/577836825 learned_addr learned my addr 172.21.15.106:0/577836825 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.241+0000 7fcfa9200640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfa4109f80 0x7fcfa419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfa9c00640 1 -- 172.21.15.106:0/577836825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfa4109f80 msgr2=0x7fcfa419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfa9c00640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfa4109f80 0x7fcfa419f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfa9c00640 1 -- 172.21.15.106:0/577836825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfa410a940 msgr2=0x7fcfa41a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfa9c00640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfa410a940 0x7fcfa41a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfa9c00640 1 -- 172.21.15.106:0/577836825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa41a6bd0 con 0x7fcfa4108d90 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfa9c00640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfa4108d90 0x7fcfa419efb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800e9c0 tx=0x7fcf9800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfa2000640 1 -- 172.21.15.106:0/577836825 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800cd80 con 0x7fcfa4108d90 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfa2000640 1 -- 172.21.15.106:0/577836825 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf980035f0 con 0x7fcfa4108d90 2024-04-28T12:49:02.241 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa41a6ec0 con 0x7fcfa4108d90 2024-04-28T12:49:02.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfa2000640 1 -- 172.21.15.106:0/577836825 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98010690 con 0x7fcfa4108d90 2024-04-28T12:49:02.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.242+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa41a72d0 con 0x7fcfa4108d90 2024-04-28T12:49:02.242 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.243+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa41100b0 con 0x7fcfa4108d90 2024-04-28T12:49:02.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.246+0000 7fcfa2000640 1 -- 172.21.15.106:0/577836825 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fcf98004000 con 0x7fcfa4108d90 2024-04-28T12:49:02.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.247+0000 7fcfa2000640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf8c0a3bb0 0x7fcf8c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.247+0000 7fcfa2000640 1 -- 172.21.15.106:0/577836825 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fcf980d0cc0 con 0x7fcfa4108d90 2024-04-28T12:49:02.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.247+0000 7fcfa9200640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf8c0a3bb0 0x7fcf8c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.243 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.248+0000 7fcfa9200640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf8c0a3bb0 0x7fcf8c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcf90031500 tx=0x7fcf90005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:02.245 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.249+0000 7fcfa2000640 1 -- 172.21.15.106:0/577836825 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcf9809d400 con 0x7fcfa4108d90 2024-04-28T12:49:02.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:02 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/611878315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:02.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:02 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3938559348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:49:02.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.452+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfa402dd80 con 0x7fcfa4108d90 2024-04-28T12:49:02.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.455+0000 7fcfa2000640 1 -- 172.21.15.106:0/577836825 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7fcf9809cd00 con 0x7fcfa4108d90 2024-04-28T12:49:02.451 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:02.451 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:02.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.457+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf8c0a3bb0 msgr2=0x7fcf8c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.457+0000 7fcfab000640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf8c0a3bb0 0x7fcf8c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcf90031500 tx=0x7fcf90005a20 comp rx=0 tx=0).stop 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.457+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfa4108d90 msgr2=0x7fcfa419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.457+0000 7fcfab000640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfa4108d90 0x7fcfa419efb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcf9800e9c0 tx=0x7fcf9800ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.457+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 shutdown_connections 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.457+0000 7fcfab000640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcf8c0a3bb0 0x7fcf8c0a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.457+0000 7fcfab000640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfa410a940 0x7fcfa41a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.457+0000 7fcfab000640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcfa4109f80 0x7fcfa419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.457+0000 7fcfab000640 1 --2- 172.21.15.106:0/577836825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfa4108d90 0x7fcfa419efb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.458+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 >> 172.21.15.106:0/577836825 conn(0x7fcfa4077d30 msgr2=0x7fcfa410d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.458+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 shutdown_connections 2024-04-28T12:49:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.458+0000 7fcfab000640 1 -- 172.21.15.106:0/577836825 wait complete. 2024-04-28T12:49:02.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-28T12:48:54.598061+0000', 'modified': '2024-04-28T12:48:59.631117+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24578}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.129:6839/4208792665', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 4208792665}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 4208792665}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24578, 'qdb_cluster': [24578]} 2024-04-28T12:49:02.473 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-28T12:49:02.473 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-28T12:49:02.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:02 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/611878315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:02.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:02 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3938559348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:49:02.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:02 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/611878315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:02.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:02 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3938559348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:49:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.681+0000 7f62abe00640 1 -- 172.21.15.106:0/3328474959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac101950 msgr2=0x7f62ac101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.681+0000 7f62abe00640 1 --2- 172.21.15.106:0/3328474959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac101950 0x7f62ac101dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f629c009900 tx=0x7f629c02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.682+0000 7f62abe00640 1 -- 172.21.15.106:0/3328474959 shutdown_connections 2024-04-28T12:49:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.682+0000 7f62abe00640 1 --2- 172.21.15.106:0/3328474959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac101950 0x7f62ac101dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.682+0000 7f62abe00640 1 --2- 172.21.15.106:0/3328474959 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62ac102310 0x7f62ac109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.682+0000 7f62abe00640 1 --2- 172.21.15.106:0/3328474959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac102c20 0x7f62ac103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.682+0000 7f62abe00640 1 -- 172.21.15.106:0/3328474959 >> 172.21.15.106:0/3328474959 conn(0x7f62ac0fc380 msgr2=0x7f62ac0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.682+0000 7f62abe00640 1 -- 172.21.15.106:0/3328474959 shutdown_connections 2024-04-28T12:49:02.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.682+0000 7f62abe00640 1 -- 172.21.15.106:0/3328474959 wait complete. 2024-04-28T12:49:02.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62abe00640 1 Processor -- start 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62abe00640 1 -- start start 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62abe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac101950 0x7f62ac19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62abe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac102310 0x7f62ac19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62abe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62ac102c20 0x7f62ac1a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62abe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac10e2f0 con 0x7f62ac102310 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62abe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac10e170 con 0x7f62ac101950 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62abe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac10e470 con 0x7f62ac102c20 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62aa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac102310 0x7f62ac19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62aa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac102310 0x7f62ac19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40864/0 (socket says 172.21.15.106:40864) 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62aa000640 1 -- 172.21.15.106:0/222718764 learned_addr learned my addr 172.21.15.106:0/222718764 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62aaa00640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac101950 0x7f62ac19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.683+0000 7f62ab400640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62ac102c20 0x7f62ac1a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62aa000640 1 -- 172.21.15.106:0/222718764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62ac102c20 msgr2=0x7f62ac1a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62aa000640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62ac102c20 0x7f62ac1a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62aa000640 1 -- 172.21.15.106:0/222718764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac101950 msgr2=0x7f62ac19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62aa000640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac101950 0x7f62ac19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62aa000640 1 -- 172.21.15.106:0/222718764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac1a2810 con 0x7f62ac102310 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62aa000640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac102310 0x7f62ac19b130 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f629c005940 tx=0x7f629c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62a3400640 1 -- 172.21.15.106:0/222718764 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c03f030 con 0x7f62ac102310 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62abe00640 1 -- 172.21.15.106:0/222718764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac1a2aa0 con 0x7f62ac102310 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62a3400640 1 -- 172.21.15.106:0/222718764 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629c003cb0 con 0x7f62ac102310 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62abe00640 1 -- 172.21.15.106:0/222718764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac1a2f90 con 0x7f62ac102310 2024-04-28T12:49:02.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.684+0000 7f62a3400640 1 -- 172.21.15.106:0/222718764 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c049060 con 0x7f62ac102310 2024-04-28T12:49:02.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.687+0000 7f62b4599740 1 Processor -- start 2024-04-28T12:49:02.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.687+0000 7f62b4599740 1 -- start start 2024-04-28T12:49:02.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.687+0000 7f62a3400640 1 -- 172.21.15.106:0/222718764 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f629c042030 con 0x7f62ac102310 2024-04-28T12:49:02.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62b4599740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 0x5648dd2f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62b4599740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f4cb0 0x5648dd2f97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62b4599740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f9e20 0x5648dd2fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62b4599740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5648dd258600 con 0x5648dd2f9e20 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62b4599740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648dd1f5100 con 0x5648dd2f4cb0 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62b4599740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5648dd2545c0 con 0x5648dd2f42c0 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62aa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f9e20 0x5648dd2fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62ab400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 0x5648dd2f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62aa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f9e20 0x5648dd2fc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40872/0 (socket says 172.21.15.106:40872) 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62aa000640 1 -- 172.21.15.106:0/3912036544 learned_addr learned my addr 172.21.15.106:0/3912036544 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62aaa00640 1 --2- 172.21.15.106:0/3912036544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f4cb0 0x5648dd2f97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62ab400640 1 -- 172.21.15.106:0/3912036544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f4cb0 msgr2=0x5648dd2f97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62ab400640 1 --2- 172.21.15.106:0/3912036544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f4cb0 0x5648dd2f97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62ab400640 1 -- 172.21.15.106:0/3912036544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f9e20 msgr2=0x5648dd2fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:49:02.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.688+0000 7f62ab400640 1 --2- 172.21.15.106:0/3912036544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f9e20 0x5648dd2fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.689+0000 7f62ab400640 1 -- 172.21.15.106:0/3912036544 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648dcf6a2d0 con 0x5648dd2f42c0 2024-04-28T12:49:02.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.689+0000 7f62ab400640 1 --2- 172.21.15.106:0/3912036544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 0x5648dd2f46e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009f00 tx=0x7f62a4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea480db397da92d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:02.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.689+0000 7f62a3400640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f62900a3bb0 0x7f62900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.689+0000 7f629be00640 1 -- 172.21.15.106:0/3912036544 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4004d20 con 0x5648dd2f42c0 2024-04-28T12:49:02.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.689+0000 7f629be00640 1 -- 172.21.15.106:0/3912036544 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a4004030 con 0x5648dd2f42c0 2024-04-28T12:49:02.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.689+0000 7f629be00640 1 -- 172.21.15.106:0/3912036544 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4005280 con 0x5648dd2f42c0 2024-04-28T12:49:02.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.689+0000 7f62a3400640 1 -- 172.21.15.106:0/222718764 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f629c0f0dc0 con 0x7f62ac102310 2024-04-28T12:49:02.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.689+0000 7f62aaa00640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f62900a3bb0 0x7f62900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 -- 172.21.15.106:0/3912036544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 msgr2=0x5648dd2f46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 --2- 172.21.15.106:0/3912036544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 0x5648dd2f46e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009f00 tx=0x7f62a4011810 comp rx=0 tx=0).stop 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 -- 172.21.15.106:0/3912036544 shutdown_connections 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 --2- 172.21.15.106:0/3912036544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f9e20 0x5648dd2fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 --2- 172.21.15.106:0/3912036544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f4cb0 0x5648dd2f97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 --2- 172.21.15.106:0/3912036544 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 0x5648dd2f46e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 -- 172.21.15.106:0/3912036544 >> 172.21.15.106:0/3912036544 conn(0x5648dd288d10 msgr2=0x5648dd2efd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62aaa00640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f62900a3bb0 0x7f62900a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5648dd2f50b0 tx=0x7f6294005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 -- 172.21.15.106:0/3912036544 shutdown_connections 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 -- 172.21.15.106:0/3912036544 wait complete. 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.690+0000 7f62b4599740 1 -- start start 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62b4599740 1 Processor -- start 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62b4599740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f42c0 0x5648dd2fe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62b4599740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f4cb0 0x5648dd2fec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62b4599740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 0x5648dd305cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62b4599740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5648dd2fcdd0 con 0x5648dd2f42c0 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62b4599740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648dd1dff70 con 0x5648dd2f9e20 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62b4599740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5648dd2fcf50 con 0x5648dd2f4cb0 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62b4599740 10 client.0 _ll_register_callbacks cb 0x5648dd24e480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62aaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f42c0 0x5648dd2fe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62aaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f42c0 0x5648dd2fe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40882/0 (socket says 172.21.15.106:40882) 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62aa000640 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f4cb0 0x5648dd2fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62aaa00640 1 -- 172.21.15.106:0/2626419116 learned_addr learned my addr 172.21.15.106:0/2626419116 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.691+0000 7f62ab400640 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 0x5648dd305cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f62aaa00640 1 -- 172.21.15.106:0/2626419116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f4cb0 msgr2=0x5648dd2fec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f62aaa00640 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f4cb0 0x5648dd2fec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f62aaa00640 1 -- 172.21.15.106:0/2626419116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 msgr2=0x5648dd305cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f62aaa00640 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 0x5648dd305cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f62aaa00640 1 -- 172.21.15.106:0/2626419116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648dd0346b0 con 0x5648dd2f42c0 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f62aa000640 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f4cb0 0x5648dd2fec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:02.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f62ab400640 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 0x5648dd305cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f62aaa00640 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f42c0 0x5648dd2fe750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6294004600 tx=0x7f6294004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f529540103f298e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f6298c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f6298c00640 1 -- 172.21.15.106:0/2626419116 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294009e60 con 0x5648dd2f42c0 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f6298c00640 1 -- 172.21.15.106:0/2626419116 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f629401f630 con 0x5648dd2f42c0 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.692+0000 7f62b4599740 1 -- 172.21.15.106:0/2626419116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a4004030 con 0x5648dd2f42c0 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f6298c00640 1 -- 172.21.15.106:0/2626419116 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f629401f830 con 0x5648dd2f42c0 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f62b4599740 10 client.14589 fetch_fsmap learned FSMap version 16 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f62b4599740 1 -- 172.21.15.106:0/2626419116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5648dd308d80 con 0x5648dd2f42c0 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f6298c00640 1 -- 172.21.15.106:0/2626419116 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a4004030 con 0x5648dd2f42c0 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f6298c00640 1 -- 172.21.15.106:0/2626419116 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f6294027070 con 0x5648dd2f42c0 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f62b4599740 10 client.14589 fetch_fsmap finished waiting for FSMap version 16 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f62b4599740 10 client.14589 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f62b4599740 20 client.14589 populate_metadata read hostname 'smithi106' 2024-04-28T12:49:02.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f62b4599740 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5648dd30b470 0x5648dd30c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f62b4599740 4 client.14589 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.693+0000 7f62b4599740 1 -- 172.21.15.106:0/2626419116 --> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648dd266210 con 0x5648dd30b470 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.694+0000 7f62aa000640 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5648dd30b470 0x5648dd30c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.694+0000 7f62aa000640 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5648dd30b470 0x5648dd30c8e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecfd1d9a075421be server_cookie=aba0cf32e685bafb in_seq=0 out_seq=0 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.694+0000 7f6298c00640 10 client.14589 ms_handle_connect on v2:172.21.15.129:6838/4208792665 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f6298c00640 1 -- 172.21.15.106:0/2626419116 <== mds.0 v2:172.21.15.129:6838/4208792665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648dd266210 con 0x5648dd30b470 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f6298c00640 10 client.14589 handle_command_reply: tid=0 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 client.14589 shutdown 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 -- 172.21.15.106:0/2626419116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f42c0 msgr2=0x5648dd2fe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f42c0 0x5648dd2fe750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6294004600 tx=0x7f6294004730 comp rx=0 tx=0).stop 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 -- 172.21.15.106:0/2626419116 shutdown_connections 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5648dd30b470 0x5648dd30c8e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 0x5648dd305cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f4cb0 0x5648dd2fec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 --2- 172.21.15.106:0/2626419116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f42c0 0x5648dd2fe750 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 -- 172.21.15.106:0/2626419116 >> 172.21.15.106:0/2626419116 conn(0x5648dd288d10 msgr2=0x5648dd2eef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:02.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f6298c00640 0 client.14589 ms_handle_reset on v2:172.21.15.129:6838/4208792665 2024-04-28T12:49:02.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 -- 172.21.15.106:0/2626419116 shutdown_connections 2024-04-28T12:49:02.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.695+0000 7f62b4599740 1 -- 172.21.15.106:0/2626419116 wait complete. 2024-04-28T12:49:02.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.696+0000 7f62b4599740 20 client.14589 trim_cache size 0 max 16384 2024-04-28T12:49:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62b4599740 1 Processor -- start 2024-04-28T12:49:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62b4599740 1 -- start start 2024-04-28T12:49:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62b4599740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f42c0 0x5648dd2f9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62b4599740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd305c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62b4599740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f9e20 0x5648dd2f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62b4599740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5648dd30ac20 con 0x5648dd2f4cb0 2024-04-28T12:49:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62b4599740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648dd30b2f0 con 0x5648dd2f42c0 2024-04-28T12:49:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62b4599740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5648dd30b9c0 con 0x5648dd2f9e20 2024-04-28T12:49:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62ab400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f9e20 0x5648dd2f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62ab400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f9e20 0x5648dd2f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:47374/0 (socket says 172.21.15.106:47374) 2024-04-28T12:49:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.715+0000 7f62ab400640 1 -- 172.21.15.106:0/3883340062 learned_addr learned my addr 172.21.15.106:0/3883340062 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f62aa000640 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd305c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f62aaa00640 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f42c0 0x5648dd2f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f62aaa00640 1 -- 172.21.15.106:0/3883340062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f9e20 msgr2=0x5648dd2f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f62aaa00640 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f9e20 0x5648dd2f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f62aaa00640 1 -- 172.21.15.106:0/3883340062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 msgr2=0x5648dd305c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f62aaa00640 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd305c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f62aaa00640 1 -- 172.21.15.106:0/3883340062 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648dd2f93a0 con 0x5648dd2f42c0 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f62ab400640 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f9e20 0x5648dd2f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f62aaa00640 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f42c0 0x5648dd2f9720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6294000c00 tx=0x7f6294003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dac596a4595c590 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.716+0000 7f629b400640 1 -- 172.21.15.106:0/3883340062 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629401f9d0 con 0x5648dd2f42c0 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f629b400640 1 -- 172.21.15.106:0/3883340062 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629401de50 con 0x5648dd2f42c0 2024-04-28T12:49:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f629b400640 1 -- 172.21.15.106:0/3883340062 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62940297c0 con 0x5648dd2f42c0 2024-04-28T12:49:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f62b4599740 1 -- 172.21.15.106:0/3883340062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f42c0 msgr2=0x5648dd2f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f62b4599740 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f42c0 0x5648dd2f9720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6294000c00 tx=0x7f6294003e10 comp rx=0 tx=0).stop 2024-04-28T12:49:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f62b4599740 1 -- 172.21.15.106:0/3883340062 shutdown_connections 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f62b4599740 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd305c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f62b4599740 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f9e20 0x5648dd2f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f62b4599740 1 --2- 172.21.15.106:0/3883340062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f42c0 0x5648dd2f9720 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f62b4599740 1 -- 172.21.15.106:0/3883340062 >> 172.21.15.106:0/3883340062 conn(0x5648dd288d10 msgr2=0x5648dd2f5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f62b4599740 1 -- 172.21.15.106:0/3883340062 shutdown_connections 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.717+0000 7f62b4599740 1 -- 172.21.15.106:0/3883340062 wait complete. 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62b4599740 1 -- start start 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62b4599740 1 Processor -- start 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62b4599740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 0x5648dd301780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62b4599740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd301cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62b4599740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 0x5648dd304c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62b4599740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5648dd30c760 con 0x5648dd2f4cb0 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62b4599740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648dd30c090 con 0x5648dd2f9e20 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62b4599740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5648dd30ce30 con 0x5648dd2f42c0 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62b4599740 10 client.0 _ll_register_callbacks cb 0x7f6294009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.718+0000 7f62aa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd301cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62aa000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd301cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40902/0 (socket says 172.21.15.106:40902) 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62aa000640 1 -- 172.21.15.106:0/3020028752 learned_addr learned my addr 172.21.15.106:0/3020028752 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62aaa00640 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 0x5648dd301780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62ab400640 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 0x5648dd304c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62aa000640 1 -- 172.21.15.106:0/3020028752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 msgr2=0x5648dd301780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62aa000640 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 0x5648dd301780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62aa000640 1 -- 172.21.15.106:0/3020028752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 msgr2=0x5648dd304c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62aa000640 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 0x5648dd304c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62aa000640 1 -- 172.21.15.106:0/3020028752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648dd305210 con 0x5648dd2f4cb0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62aa000640 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd301cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b9880 tx=0x7f629c032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c23c4deb015c673 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f6298c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f6298c00640 1 -- 172.21.15.106:0/3020028752 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c035040 con 0x5648dd2f4cb0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f62b4599740 1 -- 172.21.15.106:0/3020028752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f629401de50 con 0x5648dd2f4cb0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.719+0000 7f6298c00640 1 -- 172.21.15.106:0/3020028752 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f629c0bb1b0 con 0x5648dd2f4cb0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f6298c00640 1 -- 172.21.15.106:0/3020028752 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f629c003e60 con 0x5648dd2f4cb0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f62b4599740 10 client.14595 fetch_fsmap learned FSMap version 16 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f62b4599740 1 -- 172.21.15.106:0/3020028752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5648dd3717a0 con 0x5648dd2f4cb0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f6298c00640 1 -- 172.21.15.106:0/3020028752 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629401de50 con 0x5648dd2f4cb0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f6298c00640 1 -- 172.21.15.106:0/3020028752 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f629c0b9520 con 0x5648dd2f4cb0 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f62b4599740 10 client.14595 fetch_fsmap finished waiting for FSMap version 16 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f62b4599740 10 client.14595 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f62b4599740 20 client.14595 populate_metadata read hostname 'smithi106' 2024-04-28T12:49:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f62b4599740 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5648dd3764c0 0x5648dd3751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f62b4599740 4 client.14595 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.720+0000 7f62b4599740 1 -- 172.21.15.106:0/3020028752 --> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] -- command(tid 0: {"prefix": "status"}) -- 0x5648dd260080 con 0x5648dd3764c0 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f62aaa00640 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5648dd3764c0 0x5648dd3751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f62aaa00640 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5648dd3764c0 0x5648dd3751d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7f33a9d6350f807 server_cookie=20c555edf4040410 in_seq=0 out_seq=0 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f6298c00640 10 client.14595 ms_handle_connect on v2:172.21.15.129:6838/4208792665 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f6298c00640 1 -- 172.21.15.106:0/3020028752 <== mds.0 v2:172.21.15.129:6838/4208792665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5648dd260080 con 0x5648dd3764c0 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f6298c00640 10 client.14595 handle_command_reply: tid=0 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f62b4599740 1 client.14595 shutdown 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f62b4599740 1 -- 172.21.15.106:0/3020028752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 msgr2=0x5648dd301cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f62b4599740 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd301cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b9880 tx=0x7f629c032f50 comp rx=0 tx=0).stop 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f62b4599740 1 -- 172.21.15.106:0/3020028752 shutdown_connections 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.721+0000 7f62b4599740 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5648dd3764c0 0x5648dd3751d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62b4599740 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5648dd2f4cb0 0x5648dd301cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62b4599740 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648dd2f9e20 0x5648dd304c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f6298c00640 0 client.14595 ms_handle_reset on v2:172.21.15.129:6838/4208792665 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62b4599740 1 --2- 172.21.15.106:0/3020028752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5648dd2f42c0 0x5648dd301780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62b4599740 1 -- 172.21.15.106:0/3020028752 >> 172.21.15.106:0/3020028752 conn(0x5648dd288d10 msgr2=0x5648dd2f60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62b4599740 1 -- 172.21.15.106:0/3020028752 shutdown_connections 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62b4599740 1 -- 172.21.15.106:0/3020028752 wait complete. 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62b4599740 20 client.14595 trim_cache size 0 max 16384 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-28T12:49:02.724 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24578, 2024-04-28T12:49:02.725 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:49:02.725 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:49:02.725 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:49:02.725 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 7.6174981940000004, 2024-04-28T12:49:02.725 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 16, 2024-04-28T12:49:02.725 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 89, 2024-04-28T12:49:02.725 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:49:02.725 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 13.977296666000001 2024-04-28T12:49:02.725 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:49:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 -- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f62900a3bb0 msgr2=0x7f62900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f62900a3bb0 0x7f62900a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5648dd2f50b0 tx=0x7f6294005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 -- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac102310 msgr2=0x7f62ac19b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac102310 0x7f62ac19b130 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f629c005940 tx=0x7f629c0042f0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 -- 172.21.15.106:0/222718764 shutdown_connections 2024-04-28T12:49:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f62900a3bb0 0x7f62900a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62ac102310 0x7f62ac19b130 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62ac102c20 0x7f62ac1a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 --2- 172.21.15.106:0/222718764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62ac101950 0x7f62ac19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.722+0000 7f62abe00640 1 -- 172.21.15.106:0/222718764 >> 172.21.15.106:0/222718764 conn(0x7f62ac0fc380 msgr2=0x7f62ac10a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.723+0000 7f62abe00640 1 -- 172.21.15.106:0/222718764 shutdown_connections 2024-04-28T12:49:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:02.723+0000 7f62abe00640 1 -- 172.21.15.106:0/222718764 wait complete. 2024-04-28T12:49:02.737 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T12:49:02.737 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:49:02.737 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-28T12:49:03.171 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:03.476 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:03 smithi106 ceph-mon[28081]: pgmap v322: 129 pgs: 106 active+clean, 18 creating+peering, 5 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 832 B/s wr, 7 op/s 2024-04-28T12:49:03.476 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:03 smithi106 ceph-mon[28081]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-28T12:49:03.476 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:03 smithi106 ceph-mon[28081]: Cluster is now healthy 2024-04-28T12:49:03.476 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:03 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/577836825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:03 smithi171 ceph-mon[63127]: pgmap v322: 129 pgs: 106 active+clean, 18 creating+peering, 5 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 832 B/s wr, 7 op/s 2024-04-28T12:49:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:03 smithi171 ceph-mon[63127]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-28T12:49:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:03 smithi171 ceph-mon[63127]: Cluster is now healthy 2024-04-28T12:49:03.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:03 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/577836825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:03 smithi129 ceph-mon[62418]: pgmap v322: 129 pgs: 106 active+clean, 18 creating+peering, 5 unknown; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 832 B/s wr, 7 op/s 2024-04-28T12:49:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:03 smithi129 ceph-mon[62418]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-28T12:49:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:03 smithi129 ceph-mon[62418]: Cluster is now healthy 2024-04-28T12:49:03.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:03 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/577836825' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.436+0000 7fce03c00700 1 -- 172.21.15.106:0/2524894361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc105640 msgr2=0x7fcdfc105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.436+0000 7fce03c00700 1 --2- 172.21.15.106:0/2524894361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc105640 0x7fcdfc105ac0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009990 tx=0x7fcde4009ca0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.436+0000 7fce03c00700 1 -- 172.21.15.106:0/2524894361 shutdown_connections 2024-04-28T12:49:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.436+0000 7fce03c00700 1 --2- 172.21.15.106:0/2524894361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdfc106000 0x7fcdfc112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.436+0000 7fce03c00700 1 --2- 172.21.15.106:0/2524894361 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdfc10b640 0x7fcdfc10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.436+0000 7fce03c00700 1 --2- 172.21.15.106:0/2524894361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc105640 0x7fcdfc105ac0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.436+0000 7fce03c00700 1 -- 172.21.15.106:0/2524894361 >> 172.21.15.106:0/2524894361 conn(0x7fcdfc101130 msgr2=0x7fcdfc103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.436+0000 7fce03c00700 1 -- 172.21.15.106:0/2524894361 shutdown_connections 2024-04-28T12:49:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.436+0000 7fce03c00700 1 -- 172.21.15.106:0/2524894361 wait complete. 2024-04-28T12:49:05.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.438+0000 7fce03c00700 1 Processor -- start 2024-04-28T12:49:05.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.438+0000 7fce03c00700 1 -- start start 2024-04-28T12:49:05.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.438+0000 7fce03c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdfc105640 0x7fcdfc19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:05.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.438+0000 7fce03c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdfc106000 0x7fcdfc19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:05.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.438+0000 7fce03c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc10b640 0x7fcdfc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.438+0000 7fce03c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc115890 con 0x7fcdfc10b640 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.438+0000 7fce03c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc115710 con 0x7fcdfc105640 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce03c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc115a10 con 0x7fcdfc106000 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc10b640 0x7fcdfc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdfc105640 0x7fcdfc19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fcdf3e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdfc106000 0x7fcdfc19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc10b640 0x7fcdfc1a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:40920/0 (socket says 172.21.15.106:40920) 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fcdf3e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdfc106000 0x7fcdfc19c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:47386/0 (socket says 172.21.15.106:47386) 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01c00700 1 -- 172.21.15.106:0/1900547890 learned_addr learned my addr 172.21.15.106:0/1900547890 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdfc105640 0x7fcdfc19bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:58460/0 (socket says 172.21.15.106:58460) 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01200700 1 -- 172.21.15.106:0/1900547890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdfc106000 msgr2=0x7fcdfc19c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01200700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdfc106000 0x7fcdfc19c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01200700 1 -- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc10b640 msgr2=0x7fcdfc1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01200700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc10b640 0x7fcdfc1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01200700 1 -- 172.21.15.106:0/1900547890 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdfc1a0d30 con 0x7fcdfc105640 2024-04-28T12:49:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.439+0000 7fce01c00700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc10b640 0x7fcdfc1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.440+0000 7fce01200700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdfc105640 0x7fcdfc19bd20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00e9f0 tx=0x7fcdec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.440+0000 7fcdfa000700 1 -- 172.21.15.106:0/1900547890 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec004220 con 0x7fcdfc105640 2024-04-28T12:49:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.440+0000 7fcdfa000700 1 -- 172.21.15.106:0/1900547890 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdec0043c0 con 0x7fcdfc105640 2024-04-28T12:49:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.440+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdfc1a1020 con 0x7fcdfc105640 2024-04-28T12:49:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.440+0000 7fcdfa000700 1 -- 172.21.15.106:0/1900547890 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec005020 con 0x7fcdfc105640 2024-04-28T12:49:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.440+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdfc1a88a0 con 0x7fcdfc105640 2024-04-28T12:49:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.441+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdfc06a110 con 0x7fcdfc105640 2024-04-28T12:49:05.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.443+0000 7fcdfa000700 1 -- 172.21.15.106:0/1900547890 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fcdec004750 con 0x7fcdfc105640 2024-04-28T12:49:05.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.444+0000 7fcdfa000700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcde00a3cc0 0x7fcde00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:05.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.445+0000 7fcdfa000700 1 -- 172.21.15.106:0/1900547890 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fcdec0d0500 con 0x7fcdfc105640 2024-04-28T12:49:05.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.445+0000 7fcdf3e00700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcde00a3cc0 0x7fcde00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:05.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.445+0000 7fcdf3e00700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcde00a3cc0 0x7fcde00a6180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc103710 tx=0x7fcde400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:05.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.449+0000 7fcdfa000700 1 -- 172.21.15.106:0/1900547890 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcdec09cc40 con 0x7fcdfc105640 2024-04-28T12:49:05.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:05 smithi171 ceph-mon[63127]: pgmap v323: 129 pgs: 118 active+clean, 11 creating+peering; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-04-28T12:49:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:05 smithi129 ceph-mon[62418]: pgmap v323: 129 pgs: 118 active+clean, 11 creating+peering; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-04-28T12:49:05.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:05 smithi106 ceph-mon[28081]: pgmap v323: 129 pgs: 118 active+clean, 11 creating+peering; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 5 op/s 2024-04-28T12:49:05.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.727+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcdfc02e490 con 0x7fcdfc105640 2024-04-28T12:49:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.728+0000 7fcdfa000700 1 -- 172.21.15.106:0/1900547890 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7fcdec09c540 con 0x7fcdfc105640 2024-04-28T12:49:05.724 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-28T12:49:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.731+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcde00a3cc0 msgr2=0x7fcde00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:05.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcde00a3cc0 0x7fcde00a6180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc103710 tx=0x7fcde400b260 comp rx=0 tx=0).stop 2024-04-28T12:49:05.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdfc105640 msgr2=0x7fcdfc19bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:05.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdfc105640 0x7fcdfc19bd20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00e9f0 tx=0x7fcdec00ed00 comp rx=0 tx=0).stop 2024-04-28T12:49:05.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 shutdown_connections 2024-04-28T12:49:05.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdfc10b640 0x7fcdfc1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdfc105640 0x7fcdfc19bd20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcde00a3cc0 0x7fcde00a6180 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 --2- 172.21.15.106:0/1900547890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdfc106000 0x7fcdfc19c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:05.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 >> 172.21.15.106:0/1900547890 conn(0x7fcdfc101130 msgr2=0x7fcdfc101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:05.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 shutdown_connections 2024-04-28T12:49:05.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:05.732+0000 7fce03c00700 1 -- 172.21.15.106:0/1900547890 wait complete. 2024-04-28T12:49:06.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:06 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1900547890' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-28T12:49:06.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:06 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1900547890' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-28T12:49:06.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:06 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1900547890' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-28T12:49:06.650 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-28T12:49:06.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T12:49:06.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-28T12:49:06.651 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-28T12:49:06.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:49:06.651 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-28T12:49:06.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T12:49:06.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T12:49:06.651 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-28T12:49:06.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:06.651 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-28T12:49:06.708 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:49:06.708 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-28T12:49:06.709 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T12:49:06.710 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 41830sec preferred_lft 41830sec 2024-04-28T12:49:06.710 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-28T12:49:06.710 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:49:06.711 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-28T12:49:06.711 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:49:06.711 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:49:06.711 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add name ceph-brx type bridge 2024-04-28T12:49:06.711 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr flush dev ceph-brx 2024-04-28T12:49:06.711 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set ceph-brx up 2024-04-28T12:49:06.711 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-28T12:49:06.712 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:49:06.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:49:07.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:49:07.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:07.227 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T12:49:07.292 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-28T12:49:07.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:07.296 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-28T12:49:07.343 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-28T12:49:07.343 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T12:49:07.343 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T12:49:07.343 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T12:49:07.343 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T12:49:07.344 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:49:07.344 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:49:07.344 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T12:49:07.344 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T12:49:07.344 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T12:49:07.344 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:49:07.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:49:07.454 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:07 smithi106 ceph-mon[28081]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 518 B/s wr, 4 op/s 2024-04-28T12:49:07.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:07 smithi171 ceph-mon[63127]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 518 B/s wr, 4 op/s 2024-04-28T12:49:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:07 smithi129 ceph-mon[62418]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 518 B/s wr, 4 op/s 2024-04-28T12:49:08.028 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:49:08.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:08.037 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-28T12:49:08.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:08.085 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-28T12:49:08.131 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:49:08.132 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:49:08.132 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:49:08.132 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-28T12:49:08.132 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:49:08.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:49:08.277 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:08 smithi106 ceph-mon[28081]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 115 B/s wr, 2 op/s 2024-04-28T12:49:08.288 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:49:08.297 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-28T12:49:08.298 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:49:08.298 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:49:08.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-28T12:49:08.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-28T12:49:08.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-28T12:49:08.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-28T12:49:08.298 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-28T12:49:08.298 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:49:08.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:49:08.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:08 smithi171 ceph-mon[63127]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 115 B/s wr, 2 op/s 2024-04-28T12:49:08.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:08 smithi129 ceph-mon[62418]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 115 B/s wr, 2 op/s 2024-04-28T12:49:08.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:49:08.579 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:49:08.580 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:49:08.580 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-28T12:49:08.580 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.0 master ceph-brx 2024-04-28T12:49:08.580 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:49:08.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:49:08.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:49:08.766 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-28T12:49:08.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:49:08.766 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:49:08.808 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-28T12:49:08.809 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:49:08.809 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:49:08.895 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-28T12:49:08.971 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:49:09.021 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-28T12:49:09.021 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-28T12:49:09.021 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-28T12:49:09.021 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-28T12:49:09.021 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-28T12:49:09.021 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-28T12:49:09.022 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/ac4d72533c8d4c75e589ec2b4e9a0398da028102573e94cec3b6f47b08732ac3/merged 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/e22ccb9140fc6ad0afc85f2de31f607d473c150f523fe19fc08d9336c970053e/merged 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/0 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/7b7c5377c280fcd8fc438808232e77774b0e6a435ac878d99c0ea2cc386ac3c5/merged 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/5367c6b4402b57d7da7ea785a479156a30310285345c539cd2abbf6d8bb5baee/merged 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/86617fe81ed37bb7eea08a254a08231dfafe06b15c2a41d070b828e698f484ba/merged 2024-04-28T12:49:09.023 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/7f9e8d3a6af4e5e8446926fe2ac62b5ed1d306ba31a4d110b380f98b4afb8d87/merged 2024-04-28T12:49:09.024 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/e462835f41dcbbf32c13de623e837c9a16d049b0cb89de9ec44dfc90b0cc3e35/merged 2024-04-28T12:49:09.024 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/3a05581e95307678bbd0efb01f46f394242785f9dd196199bfa05bb2c4bab996/merged 2024-04-28T12:49:09.024 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/951919cb34972a5b0239a5de98e8a5d570ab3a1ed2a6c730a31fcf709a41449f/merged 2024-04-28T12:49:09.024 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/1045fd3b5061626e4f76bced914d592ffa3ee467419640255b614c0370a03535/merged 2024-04-28T12:49:09.024 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-28T12:49:09.024 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:49:09.024 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:49:09.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:09.025 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-28T12:49:09.071 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-28T12:49:09.071 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-28T12:49:09.119 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-28T12:49:09.186 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:49:09.228 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-28T12:49:09.228 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-28T12:49:09.228 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-28T12:49:09.228 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-28T12:49:09.228 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-28T12:49:09.228 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-28T12:49:09.228 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-28T12:49:09.228 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-28T12:49:09.228 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:49:09.229 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/ac4d72533c8d4c75e589ec2b4e9a0398da028102573e94cec3b6f47b08732ac3/merged 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/e22ccb9140fc6ad0afc85f2de31f607d473c150f523fe19fc08d9336c970053e/merged 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/0 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/7b7c5377c280fcd8fc438808232e77774b0e6a435ac878d99c0ea2cc386ac3c5/merged 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/5367c6b4402b57d7da7ea785a479156a30310285345c539cd2abbf6d8bb5baee/merged 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/86617fe81ed37bb7eea08a254a08231dfafe06b15c2a41d070b828e698f484ba/merged 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/7f9e8d3a6af4e5e8446926fe2ac62b5ed1d306ba31a4d110b380f98b4afb8d87/merged 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/e462835f41dcbbf32c13de623e837c9a16d049b0cb89de9ec44dfc90b0cc3e35/merged 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/3a05581e95307678bbd0efb01f46f394242785f9dd196199bfa05bb2c4bab996/merged 2024-04-28T12:49:09.230 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/951919cb34972a5b0239a5de98e8a5d570ab3a1ed2a6c730a31fcf709a41449f/merged 2024-04-28T12:49:09.231 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/1045fd3b5061626e4f76bced914d592ffa3ee467419640255b614c0370a03535/merged 2024-04-28T12:49:09.231 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-28T12:49:09.231 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:49:09.231 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:49:09.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:09.232 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-28T12:49:09.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi106.stderr:2024-04-28T12:49:09.429+0000 7f9051614480 -1 init, newargv = 0x561a032132e0 newargc=15 2024-04-28T12:49:09.425 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi106.stderr:ceph-fuse[71428]: starting ceph client 2024-04-28T12:49:09.436 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi106.stderr:ceph-fuse[71428]: starting fuse 2024-04-28T12:49:10.424 DEBUG:teuthology.orchestra.run.smithi106:> sudo modprobe fuse 2024-04-28T12:49:10.515 DEBUG:teuthology.orchestra.run.smithi106:> cat /proc/self/mounts | awk '{print $2}' 2024-04-28T12:49:10.569 INFO:teuthology.orchestra.run.smithi106.stdout:/proc 2024-04-28T12:49:10.569 INFO:teuthology.orchestra.run.smithi106.stdout:/sys 2024-04-28T12:49:10.569 INFO:teuthology.orchestra.run.smithi106.stdout:/dev 2024-04-28T12:49:10.569 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/security 2024-04-28T12:49:10.569 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/shm 2024-04-28T12:49:10.569 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/pts 2024-04-28T12:49:10.569 INFO:teuthology.orchestra.run.smithi106.stdout:/run 2024-04-28T12:49:10.569 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/cgroup 2024-04-28T12:49:10.569 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/pstore 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/bpf 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/ 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/selinux 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/mqueue 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/dev/hugepages 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/debug 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/tracing 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/fs/fuse/connections 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/sys/kernel/config 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-sysctl.service 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-28T12:49:10.570 INFO:teuthology.orchestra.run.smithi106.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/nfs/rpc_pipefs 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/1000 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/ceph 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/ac4d72533c8d4c75e589ec2b4e9a0398da028102573e94cec3b6f47b08732ac3/merged 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/e22ccb9140fc6ad0afc85f2de31f607d473c150f523fe19fc08d9336c970053e/merged 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/run/user/0 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/proc/sys/fs/binfmt_misc 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/7b7c5377c280fcd8fc438808232e77774b0e6a435ac878d99c0ea2cc386ac3c5/merged 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/5367c6b4402b57d7da7ea785a479156a30310285345c539cd2abbf6d8bb5baee/merged 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/86617fe81ed37bb7eea08a254a08231dfafe06b15c2a41d070b828e698f484ba/merged 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/7f9e8d3a6af4e5e8446926fe2ac62b5ed1d306ba31a4d110b380f98b4afb8d87/merged 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/e462835f41dcbbf32c13de623e837c9a16d049b0cb89de9ec44dfc90b0cc3e35/merged 2024-04-28T12:49:10.571 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/3a05581e95307678bbd0efb01f46f394242785f9dd196199bfa05bb2c4bab996/merged 2024-04-28T12:49:10.572 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/951919cb34972a5b0239a5de98e8a5d570ab3a1ed2a6c730a31fcf709a41449f/merged 2024-04-28T12:49:10.572 INFO:teuthology.orchestra.run.smithi106.stdout:/var/lib/containers/storage/overlay/1045fd3b5061626e4f76bced914d592ffa3ee467419640255b614c0370a03535/merged 2024-04-28T12:49:10.572 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns 2024-04-28T12:49:10.572 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:49:10.572 INFO:teuthology.orchestra.run.smithi106.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:49:10.572 INFO:teuthology.orchestra.run.smithi106.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-28T12:49:10.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:10.573 DEBUG:teuthology.orchestra.run.smithi106:> ls /sys/fs/fuse/connections 2024-04-28T12:49:10.623 INFO:teuthology.orchestra.run.smithi106.stdout:114 2024-04-28T12:49:10.623 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-04-28T12:49:10.623 DEBUG:teuthology.orchestra.run.smithi106:> sudo stdin-killer -- python3 -c ' 2024-04-28T12:49:10.623 DEBUG:teuthology.orchestra.run.smithi106:> import glob 2024-04-28T12:49:10.623 DEBUG:teuthology.orchestra.run.smithi106:> import re 2024-04-28T12:49:10.623 DEBUG:teuthology.orchestra.run.smithi106:> import os 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> import subprocess 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> def _find_admin_socket(client_name): 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> files = glob.glob(asok_path) 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> # Given a non-glob path, it better be there 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> if "*" not in asok_path: 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> assert(len(files) == 1) 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> return files[0] 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> for f in files: 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-28T12:49:10.624 DEBUG:teuthology.orchestra.run.smithi106:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-28T12:49:10.625 DEBUG:teuthology.orchestra.run.smithi106:> contents = proc_f.read() 2024-04-28T12:49:10.625 DEBUG:teuthology.orchestra.run.smithi106:> if mountpoint in contents: 2024-04-28T12:49:10.625 DEBUG:teuthology.orchestra.run.smithi106:> return f 2024-04-28T12:49:10.625 DEBUG:teuthology.orchestra.run.smithi106:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-28T12:49:10.625 DEBUG:teuthology.orchestra.run.smithi106:> 2024-04-28T12:49:10.625 DEBUG:teuthology.orchestra.run.smithi106:> print(_find_admin_socket("client.admin")) 2024-04-28T12:49:10.625 DEBUG:teuthology.orchestra.run.smithi106:> ' 2024-04-28T12:49:10.789 INFO:teuthology.orchestra.run.smithi106.stdout:/var/run/ceph/ceph-client.admin.71428.asok 2024-04-28T12:49:10.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:49:10.806 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.71428.asok 2024-04-28T12:49:10.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:10.807 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.71428.asok status 2024-04-28T12:49:11.039 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:49:11.039 INFO:teuthology.orchestra.run.smithi106.stdout: "metadata": { 2024-04-28T12:49:11.039 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-28T12:49:11.039 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-28T12:49:11.039 INFO:teuthology.orchestra.run.smithi106.stdout: "entity_id": "admin", 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "hostname": "smithi106", 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "pid": "71428", 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "root": "/" 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_count": 0, 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "dentry_pinned_count": 0, 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24704, 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "inst": { 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "name": { 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "client", 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "num": 24704 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-28T12:49:11.040 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 1196685491 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": { 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "v1", 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "addr": "192.168.0.1:0", 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "nonce": 1196685491 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "inst_str": "client.24704 192.168.0.1:0/1196685491", 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "addr_str": "192.168.0.1:0/1196685491", 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "inode_count": 1, 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_epoch": 16, 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch": 89, 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_epoch_barrier": 0, 2024-04-28T12:49:11.041 INFO:teuthology.orchestra.run.smithi106.stdout: "blocklisted": false, 2024-04-28T12:49:11.042 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs" 2024-04-28T12:49:11.042 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:49:11.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:11.054 DEBUG:teuthology.orchestra.run.smithi106:> stat --file-system '--printf=%T 2024-04-28T12:49:11.055 DEBUG:teuthology.orchestra.run.smithi106:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:49:11.103 INFO:teuthology.orchestra.run.smithi106.stdout:fuseblk 2024-04-28T12:49:11.103 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:49:11.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:11.104 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:49:11.232 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-28T12:49:11.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T12:49:11.377 INFO:teuthology.orchestra.run.smithi106.stdout:# file: . 2024-04-28T12:49:11.377 INFO:teuthology.orchestra.run.smithi106.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-28T12:49:11.377 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:11.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:49:11.388 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:49:11.388 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:49:11.442 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:49:11.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:11 smithi171 ceph-mon[63127]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 1 op/s 2024-04-28T12:49:11.510 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:11 smithi106 ceph-mon[28081]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 1 op/s 2024-04-28T12:49:11.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:11 smithi129 ceph-mon[62418]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 1 op/s 2024-04-28T12:49:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.648+0000 7f4325400640 1 -- 172.21.15.106:0/548872089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320106720 msgr2=0x7f432010cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.648+0000 7f4325400640 1 --2- 172.21.15.106:0/548872089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320106720 0x7f432010cfb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4318009900 tx=0x7f431802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.648+0000 7f4325400640 1 -- 172.21.15.106:0/548872089 shutdown_connections 2024-04-28T12:49:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.648+0000 7f4325400640 1 --2- 172.21.15.106:0/548872089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320106720 0x7f432010cfb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.648+0000 7f4325400640 1 --2- 172.21.15.106:0/548872089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4320105d60 0x7f43201061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.648+0000 7f4325400640 1 --2- 172.21.15.106:0/548872089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4320104b70 0x7f4320104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.648+0000 7f4325400640 1 -- 172.21.15.106:0/548872089 >> 172.21.15.106:0/548872089 conn(0x7f4320100120 msgr2=0x7f4320102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.648+0000 7f4325400640 1 -- 172.21.15.106:0/548872089 shutdown_connections 2024-04-28T12:49:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.648+0000 7f4325400640 1 -- 172.21.15.106:0/548872089 wait complete. 2024-04-28T12:49:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f4325400640 1 Processor -- start 2024-04-28T12:49:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f4325400640 1 -- start start 2024-04-28T12:49:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f4325400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4320104b70 0x7f432007a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f4325400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320105d60 0x7f432007ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f4325400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4320106720 0x7f4320077250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f4325400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f43201106f0 con 0x7f4320104b70 2024-04-28T12:49:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f4325400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4320110570 con 0x7f4320105d60 2024-04-28T12:49:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f4325400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4320110870 con 0x7f4320106720 2024-04-28T12:49:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f431de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320105d60 0x7f432007ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f431e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4320104b70 0x7f432007a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f431de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320105d60 0x7f432007ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:37758/0 (socket says 172.21.15.106:37758) 2024-04-28T12:49:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.649+0000 7f431de00640 1 -- 172.21.15.106:0/4025843242 learned_addr learned my addr 172.21.15.106:0/4025843242 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.650+0000 7f431f200640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4320106720 0x7f4320077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:11.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.650+0000 7f431de00640 1 -- 172.21.15.106:0/4025843242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4320106720 msgr2=0x7f4320077250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:11.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.650+0000 7f431de00640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4320106720 0x7f4320077250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.650+0000 7f431de00640 1 -- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4320104b70 msgr2=0x7f432007a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:11.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.650+0000 7f431de00640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4320104b70 0x7f432007a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.650+0000 7f431de00640 1 -- 172.21.15.106:0/4025843242 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4320077b10 con 0x7f4320105d60 2024-04-28T12:49:11.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.650+0000 7f431e800640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4320104b70 0x7f432007a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:11.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.650+0000 7f431f200640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4320106720 0x7f4320077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:49:11.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.650+0000 7f431de00640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320105d60 0x7f432007ac20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f430800e9f0 tx=0x7f430800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:11.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.651+0000 7f4317400640 1 -- 172.21.15.106:0/4025843242 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430800cd80 con 0x7f4320105d60 2024-04-28T12:49:11.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.651+0000 7f4317400640 1 -- 172.21.15.106:0/4025843242 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43080054d0 con 0x7f4320105d60 2024-04-28T12:49:11.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.651+0000 7f4317400640 1 -- 172.21.15.106:0/4025843242 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4308010690 con 0x7f4320105d60 2024-04-28T12:49:11.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.651+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43201a7190 con 0x7f4320105d60 2024-04-28T12:49:11.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.651+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43201a75a0 con 0x7f4320105d60 2024-04-28T12:49:11.648 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.652+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f432010be90 con 0x7f4320105d60 2024-04-28T12:49:11.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.654+0000 7f4317400640 1 -- 172.21.15.106:0/4025843242 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4308004e30 con 0x7f4320105d60 2024-04-28T12:49:11.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.655+0000 7f4317400640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f43040a3c30 0x7f43040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:11.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.656+0000 7f4317400640 1 -- 172.21.15.106:0/4025843242 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f43080d0ce0 con 0x7f4320105d60 2024-04-28T12:49:11.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.656+0000 7f431e800640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f43040a3c30 0x7f43040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:11.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.656+0000 7f431e800640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f43040a3c30 0x7f43040a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43100099a0 tx=0x7f4310009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:11.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.658+0000 7f4317400640 1 -- 172.21.15.106:0/4025843242 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4308054060 con 0x7f4320105d60 2024-04-28T12:49:11.857 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.861+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f432002dca0 con 0x7f4320105d60 2024-04-28T12:49:11.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.863+0000 7f4317400640 1 -- 172.21.15.106:0/4025843242 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7f430809d420 con 0x7f4320105d60 2024-04-28T12:49:11.859 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:11.860 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f43040a3c30 msgr2=0x7f43040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f43040a3c30 0x7f43040a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f43100099a0 tx=0x7f4310009500 comp rx=0 tx=0).stop 2024-04-28T12:49:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320105d60 msgr2=0x7f432007ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320105d60 0x7f432007ac20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f430800e9f0 tx=0x7f430800edf0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 shutdown_connections 2024-04-28T12:49:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f43040a3c30 0x7f43040a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4320106720 0x7f4320077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4320105d60 0x7f432007ac20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.862 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 --2- 172.21.15.106:0/4025843242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4320104b70 0x7f432007a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:11.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 >> 172.21.15.106:0/4025843242 conn(0x7f4320100120 msgr2=0x7f4320109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:11.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 shutdown_connections 2024-04-28T12:49:11.863 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:11.866+0000 7f4325400640 1 -- 172.21.15.106:0/4025843242 wait complete. 2024-04-28T12:49:11.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:48:52.916511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:48:53.528808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:48:57.154563+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:49:11.881 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:49:12.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.084+0000 7f42b3e00640 1 -- 172.21.15.106:0/3230223575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4103a50 msgr2=0x7f42b4103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.084+0000 7f42b3e00640 1 --2- 172.21.15.106:0/3230223575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4103a50 0x7f42b4103ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009900 tx=0x7f42a002a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.084+0000 7f42b3e00640 1 -- 172.21.15.106:0/3230223575 shutdown_connections 2024-04-28T12:49:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.084+0000 7f42b3e00640 1 --2- 172.21.15.106:0/3230223575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4104410 0x7f42b410aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.084+0000 7f42b3e00640 1 --2- 172.21.15.106:0/3230223575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4103a50 0x7f42b4103ed0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.084+0000 7f42b3e00640 1 --2- 172.21.15.106:0/3230223575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4102860 0x7f42b4102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.084+0000 7f42b3e00640 1 -- 172.21.15.106:0/3230223575 >> 172.21.15.106:0/3230223575 conn(0x7f42b40fddd0 msgr2=0x7f42b4100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.084+0000 7f42b3e00640 1 -- 172.21.15.106:0/3230223575 shutdown_connections 2024-04-28T12:49:12.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.084+0000 7f42b3e00640 1 -- 172.21.15.106:0/3230223575 wait complete. 2024-04-28T12:49:12.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.085+0000 7f42b3e00640 1 Processor -- start 2024-04-28T12:49:12.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.085+0000 7f42b3e00640 1 -- start start 2024-04-28T12:49:12.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4102860 0x7f42b419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4103a50 0x7f42b419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4104410 0x7f42b41a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b3e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f42b410e3c0 con 0x7f42b4103a50 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b3e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f42b410e240 con 0x7f42b4102860 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b3e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42b410e540 con 0x7f42b4104410 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b2000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4103a50 0x7f42b419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b2000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4103a50 0x7f42b419b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58250/0 (socket says 172.21.15.106:58250) 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b2a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4102860 0x7f42b419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b2000640 1 -- 172.21.15.106:0/177389060 learned_addr learned my addr 172.21.15.106:0/177389060 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.086+0000 7f42b3400640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4104410 0x7f42b41a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42b2000640 1 -- 172.21.15.106:0/177389060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4104410 msgr2=0x7f42b41a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42b2000640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4104410 0x7f42b41a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42b2000640 1 -- 172.21.15.106:0/177389060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4102860 msgr2=0x7f42b419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42b2000640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4102860 0x7f42b419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42b2000640 1 -- 172.21.15.106:0/177389060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b41a28f0 con 0x7f42b4103a50 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42b3400640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4104410 0x7f42b41a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:12.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42b2000640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4103a50 0x7f42b419b1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f42a0005ab0 tx=0x7f42a0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:12.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42ab400640 1 -- 172.21.15.106:0/177389060 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a003f070 con 0x7f42b4103a50 2024-04-28T12:49:12.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b41a2b80 con 0x7f42b4103a50 2024-04-28T12:49:12.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42ab400640 1 -- 172.21.15.106:0/177389060 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a0031c50 con 0x7f42b4103a50 2024-04-28T12:49:12.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.087+0000 7f42ab400640 1 -- 172.21.15.106:0/177389060 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0035030 con 0x7f42b4103a50 2024-04-28T12:49:12.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.088+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b41a3070 con 0x7f42b4103a50 2024-04-28T12:49:12.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.089+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b41a3240 con 0x7f42b4103a50 2024-04-28T12:49:12.086 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.091+0000 7f42ab400640 1 -- 172.21.15.106:0/177389060 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f42a0042070 con 0x7f42b4103a50 2024-04-28T12:49:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.092+0000 7f42ab400640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3bb0 0x7f42980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.092+0000 7f42ab400640 1 -- 172.21.15.106:0/177389060 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f42a00f0d60 con 0x7f42b4103a50 2024-04-28T12:49:12.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.093+0000 7f42b2a00640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3bb0 0x7f42980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.089 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.093+0000 7f42b2a00640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3bb0 0x7f42980a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f42b419bb80 tx=0x7f42a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:12.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.097+0000 7f42ab400640 1 -- 172.21.15.106:0/177389060 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f42a00bd4a0 con 0x7f42b4103a50 2024-04-28T12:49:12.297 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.301+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42b41a3410 con 0x7f42b4103a50 2024-04-28T12:49:12.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.302+0000 7f42ab400640 1 -- 172.21.15.106:0/177389060 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f42a0031e00 con 0x7f42b4103a50 2024-04-28T12:49:12.298 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3bb0 msgr2=0x7f42980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3bb0 0x7f42980a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f42b419bb80 tx=0x7f42a4005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4103a50 msgr2=0x7f42b419b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4103a50 0x7f42b419b1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f42a0005ab0 tx=0x7f42a0003b90 comp rx=0 tx=0).stop 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 shutdown_connections 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3bb0 0x7f42980a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4103a50 0x7f42b419b1f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4104410 0x7f42b41a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 --2- 172.21.15.106:0/177389060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4102860 0x7f42b419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 >> 172.21.15.106:0/177389060 conn(0x7f42b40fddd0 msgr2=0x7f42b4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:12.301 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 shutdown_connections 2024-04-28T12:49:12.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.305+0000 7f42b3e00640 1 -- 172.21.15.106:0/177389060 wait complete. 2024-04-28T12:49:12.316 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:12 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4025843242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:12.319 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:49:12.483 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:12 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/4025843242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:12.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:12 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/4025843242' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:12.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.522+0000 7f96bf600640 1 -- 172.21.15.106:0/4180098196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 msgr2=0x7f96b8102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.522+0000 7f96bf600640 1 --2- 172.21.15.106:0/4180098196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 0x7f96b8102da0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009900 tx=0x7f96a802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.522+0000 7f96bf600640 1 -- 172.21.15.106:0/4180098196 shutdown_connections 2024-04-28T12:49:12.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.522+0000 7f96bf600640 1 --2- 172.21.15.106:0/4180098196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96b8104530 0x7f96b810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.522+0000 7f96bf600640 1 --2- 172.21.15.106:0/4180098196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96b8103b70 0x7f96b8103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.522+0000 7f96bf600640 1 --2- 172.21.15.106:0/4180098196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 0x7f96b8102da0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.522+0000 7f96bf600640 1 -- 172.21.15.106:0/4180098196 >> 172.21.15.106:0/4180098196 conn(0x7f96b80fdf10 msgr2=0x7f96b8100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:12.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.522+0000 7f96bf600640 1 -- 172.21.15.106:0/4180098196 shutdown_connections 2024-04-28T12:49:12.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.522+0000 7f96bf600640 1 -- 172.21.15.106:0/4180098196 wait complete. 2024-04-28T12:49:12.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.523+0000 7f96bf600640 1 Processor -- start 2024-04-28T12:49:12.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bf600640 1 -- start start 2024-04-28T12:49:12.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bf600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 0x7f96b819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bf600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96b8103b70 0x7f96b819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bf600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96b8104530 0x7f96b81a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bf600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f96b810e4d0 con 0x7f96b8104530 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bf600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f96b810e350 con 0x7f96b8103b70 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bf600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96b810e650 con 0x7f96b8102980 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bcc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 0x7f96b819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bd600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96b8104530 0x7f96b81a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bcc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 0x7f96b819add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:34730/0 (socket says 172.21.15.106:34730) 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96bcc00640 1 -- 172.21.15.106:0/569387275 learned_addr learned my addr 172.21.15.106:0/569387275 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.524+0000 7f96b7e00640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96b8103b70 0x7f96b819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96bcc00640 1 -- 172.21.15.106:0/569387275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96b8103b70 msgr2=0x7f96b819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96bcc00640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96b8103b70 0x7f96b819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96bcc00640 1 -- 172.21.15.106:0/569387275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96b8104530 msgr2=0x7f96b81a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96bcc00640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96b8104530 0x7f96b81a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96bcc00640 1 -- 172.21.15.106:0/569387275 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b81a29f0 con 0x7f96b8102980 2024-04-28T12:49:12.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96bcc00640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 0x7f96b819add0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f96a8031500 tx=0x7f96a8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:12.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96b5600640 1 -- 172.21.15.106:0/569387275 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a803f070 con 0x7f96b8102980 2024-04-28T12:49:12.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96b5600640 1 -- 172.21.15.106:0/569387275 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a8031c50 con 0x7f96b8102980 2024-04-28T12:49:12.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b81a2c80 con 0x7f96b8102980 2024-04-28T12:49:12.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96b5600640 1 -- 172.21.15.106:0/569387275 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8035030 con 0x7f96b8102980 2024-04-28T12:49:12.522 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.525+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b81a31c0 con 0x7f96b8102980 2024-04-28T12:49:12.523 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.526+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b81a3390 con 0x7f96b8102980 2024-04-28T12:49:12.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.528+0000 7f96b5600640 1 -- 172.21.15.106:0/569387275 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f96a8042070 con 0x7f96b8102980 2024-04-28T12:49:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.529+0000 7f96b5600640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96980a3c30 0x7f96980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.530+0000 7f96b5600640 1 -- 172.21.15.106:0/569387275 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f96a80f0d00 con 0x7f96b8102980 2024-04-28T12:49:12.525 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.530+0000 7f96b7e00640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96980a3c30 0x7f96980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.526 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.530+0000 7f96b7e00640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96980a3c30 0x7f96980a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96b819c2e0 tx=0x7f96a0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:12.529 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.534+0000 7f96b5600640 1 -- 172.21.15.106:0/569387275 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f96a80bcdf0 con 0x7f96b8102980 2024-04-28T12:49:12.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.734+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96b81a3560 con 0x7f96b8102980 2024-04-28T12:49:12.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.735+0000 7f96b5600640 1 -- 172.21.15.106:0/569387275 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f96a80c10f0 con 0x7f96b8102980 2024-04-28T12:49:12.731 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:49:12.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.737+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96980a3c30 msgr2=0x7f96980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.737+0000 7f96bf600640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96980a3c30 0x7f96980a60f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96b819c2e0 tx=0x7f96a0009500 comp rx=0 tx=0).stop 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.737+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 msgr2=0x7f96b819add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.737+0000 7f96bf600640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 0x7f96b819add0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f96a8031500 tx=0x7f96a8003c10 comp rx=0 tx=0).stop 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.738+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 shutdown_connections 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.738+0000 7f96bf600640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f96980a3c30 0x7f96980a60f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.738+0000 7f96bf600640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f96b8104530 0x7f96b81a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.738+0000 7f96bf600640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f96b8103b70 0x7f96b819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.738+0000 7f96bf600640 1 --2- 172.21.15.106:0/569387275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b8102980 0x7f96b819add0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.738+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 >> 172.21.15.106:0/569387275 conn(0x7f96b80fdf10 msgr2=0x7f96b8109330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.738+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 shutdown_connections 2024-04-28T12:49:12.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.738+0000 7f96bf600640 1 -- 172.21.15.106:0/569387275 wait complete. 2024-04-28T12:49:12.753 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:49:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.957+0000 7fd7f9000640 1 -- 172.21.15.106:0/1929282661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f4105b60 msgr2=0x7fd7f4105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.957+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1929282661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f4105b60 0x7fd7f4105fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009900 tx=0x7fd7e002a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.958+0000 7fd7f9000640 1 -- 172.21.15.106:0/1929282661 shutdown_connections 2024-04-28T12:49:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.958+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1929282661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7f4106520 0x7fd7f410cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.958+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1929282661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f4105b60 0x7fd7f4105fe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.958+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1929282661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f4104970 0x7fd7f4104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.958+0000 7fd7f9000640 1 -- 172.21.15.106:0/1929282661 >> 172.21.15.106:0/1929282661 conn(0x7fd7f40fff00 msgr2=0x7fd7f4102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:12.954 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.958+0000 7fd7f9000640 1 -- 172.21.15.106:0/1929282661 shutdown_connections 2024-04-28T12:49:12.955 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.958+0000 7fd7f9000640 1 -- 172.21.15.106:0/1929282661 wait complete. 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f9000640 1 Processor -- start 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f9000640 1 -- start start 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f9000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f4104970 0x7fd7f419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f9000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7f4105b60 0x7fd7f419b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f9000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f4106520 0x7fd7f41a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f9000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4110480 con 0x7fd7f4104970 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f9000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4110300 con 0x7fd7f4105b60 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f9000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4110600 con 0x7fd7f4106520 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f4104970 0x7fd7f419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f4104970 0x7fd7f419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58284/0 (socket says 172.21.15.106:58284) 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f3400640 1 -- 172.21.15.106:0/1066569294 learned_addr learned my addr 172.21.15.106:0/1066569294 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f3e00640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f4106520 0x7fd7f41a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.959+0000 7fd7f2a00640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7f4105b60 0x7fd7f419b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7f3400640 1 -- 172.21.15.106:0/1066569294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f4106520 msgr2=0x7fd7f41a20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7f3400640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f4106520 0x7fd7f41a20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7f3400640 1 -- 172.21.15.106:0/1066569294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7f4105b60 msgr2=0x7fd7f419b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7f3400640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7f4105b60 0x7fd7f419b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7f3400640 1 -- 172.21.15.106:0/1066569294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f41a2750 con 0x7fd7f4104970 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7f3e00640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f4106520 0x7fd7f41a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7f3400640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f4104970 0x7fd7f419ab80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e9c0 tx=0x7fd7e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7ebe00640 1 -- 172.21.15.106:0/1066569294 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400cde0 con 0x7fd7f4104970 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7ebe00640 1 -- 172.21.15.106:0/1066569294 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e40108e0 con 0x7fd7f4104970 2024-04-28T12:49:12.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f41a2a40 con 0x7fd7f4104970 2024-04-28T12:49:12.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.960+0000 7fd7ebe00640 1 -- 172.21.15.106:0/1066569294 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4018960 con 0x7fd7f4104970 2024-04-28T12:49:12.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.961+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f41a2f60 con 0x7fd7f4104970 2024-04-28T12:49:12.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.962+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f410bd60 con 0x7fd7f4104970 2024-04-28T12:49:12.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.963+0000 7fd7ebe00640 1 -- 172.21.15.106:0/1066569294 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd7e401f020 con 0x7fd7f4104970 2024-04-28T12:49:12.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.965+0000 7fd7ebe00640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7d80a3bb0 0x7fd7d80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:12.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.965+0000 7fd7ebe00640 1 -- 172.21.15.106:0/1066569294 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd7e40d0910 con 0x7fd7f4104970 2024-04-28T12:49:12.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.965+0000 7fd7f2a00640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7d80a3bb0 0x7fd7d80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:12.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.966+0000 7fd7f2a00640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7d80a3bb0 0x7fd7d80a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0000c00 tx=0x7fd7e00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:12.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:12.969+0000 7fd7ebe00640 1 -- 172.21.15.106:0/1066569294 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd7e4054060 con 0x7fd7f4104970 2024-04-28T12:49:13.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.174+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd7f402dd80 con 0x7fd7f4104970 2024-04-28T12:49:13.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.175+0000 7fd7ebe00640 1 -- 172.21.15.106:0/1066569294 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fd7e409d050 con 0x7fd7f4104970 2024-04-28T12:49:13.171 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-28T12:49:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.178+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7d80a3bb0 msgr2=0x7fd7d80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.178+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7d80a3bb0 0x7fd7d80a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0000c00 tx=0x7fd7e00314a0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.178+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f4104970 msgr2=0x7fd7f419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.178+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f4104970 0x7fd7f419ab80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e9c0 tx=0x7fd7e400ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.179+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 shutdown_connections 2024-04-28T12:49:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.179+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7d80a3bb0 0x7fd7d80a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.179+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd7f4106520 0x7fd7f41a20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.179+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7f4105b60 0x7fd7f419b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.179+0000 7fd7f9000640 1 --2- 172.21.15.106:0/1066569294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7f4104970 0x7fd7f419ab80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.179+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 >> 172.21.15.106:0/1066569294 conn(0x7fd7f40fff00 msgr2=0x7fd7f4108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:13.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.179+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 shutdown_connections 2024-04-28T12:49:13.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.179+0000 7fd7f9000640 1 -- 172.21.15.106:0/1066569294 wait complete. 2024-04-28T12:49:13.194 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:49:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.403+0000 7f1804a00640 1 -- 172.21.15.106:0/1633330524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1800106620 msgr2=0x7f180010ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.403+0000 7f1804a00640 1 --2- 172.21.15.106:0/1633330524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1800106620 0x7f180010ceb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009ca0 tx=0x7f17f802a8c0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.403+0000 7f1804a00640 1 -- 172.21.15.106:0/1633330524 shutdown_connections 2024-04-28T12:49:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.403+0000 7f1804a00640 1 --2- 172.21.15.106:0/1633330524 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1800106620 0x7f180010ceb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.403+0000 7f1804a00640 1 --2- 172.21.15.106:0/1633330524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1800105c60 0x7f18001060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.403+0000 7f1804a00640 1 --2- 172.21.15.106:0/1633330524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1800104a70 0x7f1800104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.403+0000 7f1804a00640 1 -- 172.21.15.106:0/1633330524 >> 172.21.15.106:0/1633330524 conn(0x7f18000fffe0 msgr2=0x7f1800102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.403+0000 7f1804a00640 1 -- 172.21.15.106:0/1633330524 shutdown_connections 2024-04-28T12:49:13.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.404+0000 7f1804a00640 1 -- 172.21.15.106:0/1633330524 wait complete. 2024-04-28T12:49:13.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.404+0000 7f1804a00640 1 Processor -- start 2024-04-28T12:49:13.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f1804a00640 1 -- start start 2024-04-28T12:49:13.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f1804a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1800104a70 0x7f180019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f1804a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1800105c60 0x7f180019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f1804a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1800106620 0x7f18001a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f1804a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18001105f0 con 0x7f1800106620 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f1804a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1800110470 con 0x7f1800104a70 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f1804a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1800110770 con 0x7f1800105c60 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17fe800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1800104a70 0x7f180019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17fde00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1800105c60 0x7f180019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17ff200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1800106620 0x7f18001a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17fe800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1800104a70 0x7f180019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:37828/0 (socket says 172.21.15.106:37828) 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17fe800640 1 -- 172.21.15.106:0/1082165588 learned_addr learned my addr 172.21.15.106:0/1082165588 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17fe800640 1 -- 172.21.15.106:0/1082165588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1800105c60 msgr2=0x7f180019b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17fe800640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1800105c60 0x7f180019b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17fe800640 1 -- 172.21.15.106:0/1082165588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1800106620 msgr2=0x7f18001a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17fe800640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1800106620 0x7f18001a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.405+0000 7f17fe800640 1 -- 172.21.15.106:0/1082165588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18001a2890 con 0x7f1800104a70 2024-04-28T12:49:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.406+0000 7f17fe800640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1800104a70 0x7f180019ac80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9c0 tx=0x7f17f000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.406+0000 7f17f7400640 1 -- 172.21.15.106:0/1082165588 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f000cd80 con 0x7f1800104a70 2024-04-28T12:49:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.406+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18001a2b80 con 0x7f1800104a70 2024-04-28T12:49:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.406+0000 7f17f7400640 1 -- 172.21.15.106:0/1082165588 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f00035f0 con 0x7f1800104a70 2024-04-28T12:49:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.406+0000 7f17f7400640 1 -- 172.21.15.106:0/1082165588 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0010690 con 0x7f1800104a70 2024-04-28T12:49:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.406+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18001a30d0 con 0x7f1800104a70 2024-04-28T12:49:13.404 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.407+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180010bd90 con 0x7f1800104a70 2024-04-28T12:49:13.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.409+0000 7f17f7400640 1 -- 172.21.15.106:0/1082165588 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f17f00037a0 con 0x7f1800104a70 2024-04-28T12:49:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.410+0000 7f17f7400640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17e40a3bb0 0x7f17e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:13.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.410+0000 7f17f7400640 1 -- 172.21.15.106:0/1082165588 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f17f00d0a90 con 0x7f1800104a70 2024-04-28T12:49:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.411+0000 7f17fde00640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17e40a3bb0 0x7f17e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:13.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.411+0000 7f17fde00640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17e40a3bb0 0x7f17e40a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009bd0 tx=0x7f17ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:13.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.414+0000 7f17f7400640 1 -- 172.21.15.106:0/1082165588 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f17f009d1d0 con 0x7f1800104a70 2024-04-28T12:49:13.418 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:13 smithi171 ceph-mon[63127]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2024-04-28T12:49:13.418 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:13 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/177389060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:49:13.418 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:13 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/569387275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:49:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:13 smithi129 ceph-mon[62418]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2024-04-28T12:49:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:13 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/177389060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:49:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:13 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/569387275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:49:13.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:13 smithi106 ceph-mon[28081]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2024-04-28T12:49:13.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:13 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/177389060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:49:13.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:13 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/569387275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:49:13.600 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.604+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f180002dcf0 con 0x7f1800104a70 2024-04-28T12:49:13.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.605+0000 7f17f7400640 1 -- 172.21.15.106:0/1082165588 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f17f009cad0 con 0x7f1800104a70 2024-04-28T12:49:13.601 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:49:13.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17e40a3bb0 msgr2=0x7f17e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17e40a3bb0 0x7f17e40a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009bd0 tx=0x7f17ec005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1800104a70 msgr2=0x7f180019ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1800104a70 0x7f180019ac80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f17f000e9c0 tx=0x7f17f000ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 shutdown_connections 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17e40a3bb0 0x7f17e40a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1800106620 0x7f18001a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1800105c60 0x7f180019b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 --2- 172.21.15.106:0/1082165588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1800104a70 0x7f180019ac80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.607+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 >> 172.21.15.106:0/1082165588 conn(0x7f18000fffe0 msgr2=0x7f1800108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.608+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 shutdown_connections 2024-04-28T12:49:13.604 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.608+0000 7f1804a00640 1 -- 172.21.15.106:0/1082165588 wait complete. 2024-04-28T12:49:13.621 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:49:13.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.827+0000 7f1a22800640 1 -- 172.21.15.106:0/977648914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a1c075390 msgr2=0x7f1a1c075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.827+0000 7f1a22800640 1 --2- 172.21.15.106:0/977648914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a1c075390 0x7f1a1c075810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009930 tx=0x7f1a0402a420 comp rx=0 tx=0).stop 2024-04-28T12:49:13.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.827+0000 7f1a22800640 1 -- 172.21.15.106:0/977648914 shutdown_connections 2024-04-28T12:49:13.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.827+0000 7f1a22800640 1 --2- 172.21.15.106:0/977648914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a1c075390 0x7f1a1c075810 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.827+0000 7f1a22800640 1 --2- 172.21.15.106:0/977648914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1c10ef50 0x7f1a1c111340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.827+0000 7f1a22800640 1 --2- 172.21.15.106:0/977648914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a1c076970 0x7f1a1c074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.827+0000 7f1a22800640 1 -- 172.21.15.106:0/977648914 >> 172.21.15.106:0/977648914 conn(0x7f1a1c0fdf10 msgr2=0x7f1a1c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:13.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.827+0000 7f1a22800640 1 -- 172.21.15.106:0/977648914 shutdown_connections 2024-04-28T12:49:13.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.827+0000 7f1a22800640 1 -- 172.21.15.106:0/977648914 wait complete. 2024-04-28T12:49:13.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.828+0000 7f1a22800640 1 Processor -- start 2024-04-28T12:49:13.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.828+0000 7f1a22800640 1 -- start start 2024-04-28T12:49:13.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.828+0000 7f1a22800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a1c075390 0x7f1a1c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:13.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.828+0000 7f1a22800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a1c076970 0x7f1a1c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:13.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.828+0000 7f1a22800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1c10ef50 0x7f1a1c1a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.828+0000 7f1a22800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c114a80 con 0x7f1a1c076970 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.828+0000 7f1a22800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c114900 con 0x7f1a1c10ef50 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.828+0000 7f1a22800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c114c00 con 0x7f1a1c075390 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1b400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a1c075390 0x7f1a1c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1b400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a1c075390 0x7f1a1c19f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:34782/0 (socket says 172.21.15.106:34782) 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1b400640 1 -- 172.21.15.106:0/692594625 learned_addr learned my addr 172.21.15.106:0/692594625 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1be00640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1c10ef50 0x7f1a1c1a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a12a00640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a1c076970 0x7f1a1c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1b400640 1 -- 172.21.15.106:0/692594625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1c10ef50 msgr2=0x7f1a1c1a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1b400640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1c10ef50 0x7f1a1c1a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1b400640 1 -- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a1c076970 msgr2=0x7f1a1c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1b400640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a1c076970 0x7f1a1c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1b400640 1 -- 172.21.15.106:0/692594625 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1c1a6e00 con 0x7f1a1c075390 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1be00640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1c10ef50 0x7f1a1c1a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:13.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.829+0000 7f1a1b400640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a1c075390 0x7f1a1c19f1e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00ea60 tx=0x7f1a0c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:13.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.830+0000 7f1a18c00640 1 -- 172.21.15.106:0/692594625 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c00cde0 con 0x7f1a1c075390 2024-04-28T12:49:13.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.830+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1c1a70f0 con 0x7f1a1c075390 2024-04-28T12:49:13.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.830+0000 7f1a18c00640 1 -- 172.21.15.106:0/692594625 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a0c0035f0 con 0x7f1a1c075390 2024-04-28T12:49:13.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.830+0000 7f1a18c00640 1 -- 172.21.15.106:0/692594625 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c010690 con 0x7f1a1c075390 2024-04-28T12:49:13.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.830+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1c1a7610 con 0x7f1a1c075390 2024-04-28T12:49:13.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.831+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a1c105120 con 0x7f1a1c075390 2024-04-28T12:49:13.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.833+0000 7f1a18c00640 1 -- 172.21.15.106:0/692594625 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1a0c0037a0 con 0x7f1a1c075390 2024-04-28T12:49:13.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.834+0000 7f1a18c00640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a000a3bb0 0x7f1a000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:13.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.834+0000 7f1a18c00640 1 -- 172.21.15.106:0/692594625 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1a0c0d0930 con 0x7f1a1c075390 2024-04-28T12:49:13.830 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.834+0000 7f1a12a00640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a000a3bb0 0x7f1a000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:13.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.835+0000 7f1a12a00640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a000a3bb0 0x7f1a000a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a04006010 tx=0x7f1a040314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:13.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:13.838+0000 7f1a18c00640 1 -- 172.21.15.106:0/692594625 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1a0c09d070 con 0x7f1a1c075390 2024-04-28T12:49:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.057+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1a1c02dad0 con 0x7f1a1c075390 2024-04-28T12:49:14.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.058+0000 7f1a18c00640 1 -- 172.21.15.106:0/692594625 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+599 (secure 0 0 0) 0x7f1a0c09c970 con 0x7f1a1c075390 2024-04-28T12:49:14.054 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6801/2253669851 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:14.054 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6800/2253669851 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:14.054 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/3545890663 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:14.054 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/1369056801 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:14.055 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/938762297 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:14.055 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/1467729753 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:14.055 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6800/102356115 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:14.055 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/2991131145 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:14.055 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6801/102356115 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:14.055 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/2350345760 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:14.055 INFO:teuthology.orchestra.run.smithi106.stderr:listed 10 entries 2024-04-28T12:49:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.061+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a000a3bb0 msgr2=0x7f1a000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.061+0000 7f1a22800640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a000a3bb0 0x7f1a000a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a04006010 tx=0x7f1a040314a0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.061+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a1c075390 msgr2=0x7f1a1c19f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.061+0000 7f1a22800640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a1c075390 0x7f1a1c19f1e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00ea60 tx=0x7f1a0c00ef20 comp rx=0 tx=0).stop 2024-04-28T12:49:14.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.062+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 shutdown_connections 2024-04-28T12:49:14.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.062+0000 7f1a22800640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1a000a3bb0 0x7f1a000a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.062+0000 7f1a22800640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a1c076970 0x7f1a1c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.062+0000 7f1a22800640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1c10ef50 0x7f1a1c1a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.062+0000 7f1a22800640 1 --2- 172.21.15.106:0/692594625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1a1c075390 0x7f1a1c19f1e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.062+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 >> 172.21.15.106:0/692594625 conn(0x7f1a1c0fdf10 msgr2=0x7f1a1c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:14.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.062+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 shutdown_connections 2024-04-28T12:49:14.058 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.062+0000 7f1a22800640 1 -- 172.21.15.106:0/692594625 wait complete. 2024-04-28T12:49:14.076 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:49:14.077 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:49:14.130 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:49:14.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:14 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1066569294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:49:14.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:14 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1082165588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:49:14.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:14 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/692594625' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-28T12:49:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.331+0000 7fc1b7800640 1 -- 172.21.15.106:0/3800648587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b010a180 msgr2=0x7fc1b010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.331+0000 7fc1b7800640 1 --2- 172.21.15.106:0/3800648587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b010a180 0x7fc1b010a600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009ca0 tx=0x7fc1a002a8c0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.331+0000 7fc1b7800640 1 -- 172.21.15.106:0/3800648587 shutdown_connections 2024-04-28T12:49:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.331+0000 7fc1b7800640 1 --2- 172.21.15.106:0/3800648587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b010a180 0x7fc1b010a600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.331+0000 7fc1b7800640 1 --2- 172.21.15.106:0/3800648587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1b010ab40 0x7fc1b01113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.331+0000 7fc1b7800640 1 --2- 172.21.15.106:0/3800648587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1b0108f90 0x7fc1b01093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.331+0000 7fc1b7800640 1 -- 172.21.15.106:0/3800648587 >> 172.21.15.106:0/3800648587 conn(0x7fc1b0077f40 msgr2=0x7fc1b007a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.331+0000 7fc1b7800640 1 -- 172.21.15.106:0/3800648587 shutdown_connections 2024-04-28T12:49:14.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.331+0000 7fc1b7800640 1 -- 172.21.15.106:0/3800648587 wait complete. 2024-04-28T12:49:14.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.332+0000 7fc1b7800640 1 Processor -- start 2024-04-28T12:49:14.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.332+0000 7fc1b7800640 1 -- start start 2024-04-28T12:49:14.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b7800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b0108f90 0x7fc1b019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:14.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b7800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1b010a180 0x7fc1b019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:14.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b7800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1b010ab40 0x7fc1b01a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:14.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b7800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0114b20 con 0x7fc1b0108f90 2024-04-28T12:49:14.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b7800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b01149a0 con 0x7fc1b010a180 2024-04-28T12:49:14.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b7800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b0114ca0 con 0x7fc1b010ab40 2024-04-28T12:49:14.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b4e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b0108f90 0x7fc1b019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:14.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1b010a180 0x7fc1b019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:14.330 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1b010a180 0x7fc1b019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:37862/0 (socket says 172.21.15.106:37862) 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b4e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b0108f90 0x7fc1b019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58348/0 (socket says 172.21.15.106:58348) 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1afe00640 1 -- 172.21.15.106:0/2598487167 learned_addr learned my addr 172.21.15.106:0/2598487167 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b5800640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1b010ab40 0x7fc1b01a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b4e00640 1 -- 172.21.15.106:0/2598487167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1b010ab40 msgr2=0x7fc1b01a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b4e00640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1b010ab40 0x7fc1b01a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b4e00640 1 -- 172.21.15.106:0/2598487167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1b010a180 msgr2=0x7fc1b019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.333+0000 7fc1b4e00640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1b010a180 0x7fc1b019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.334+0000 7fc1b4e00640 1 -- 172.21.15.106:0/2598487167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b01a6db0 con 0x7fc1b0108f90 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.334+0000 7fc1afe00640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1b010a180 0x7fc1b019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.334+0000 7fc1b4e00640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b0108f90 0x7fc1b019f190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e9c0 tx=0x7fc19800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.334+0000 7fc1ad600640 1 -- 172.21.15.106:0/2598487167 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19800ccc0 con 0x7fc1b0108f90 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.334+0000 7fc1ad600640 1 -- 172.21.15.106:0/2598487167 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1980035f0 con 0x7fc1b0108f90 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.334+0000 7fc1ad600640 1 -- 172.21.15.106:0/2598487167 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19800ce60 con 0x7fc1b0108f90 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.334+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b01a70a0 con 0x7fc1b0108f90 2024-04-28T12:49:14.331 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.334+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b01a75f0 con 0x7fc1b0108f90 2024-04-28T12:49:14.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.335+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b01102b0 con 0x7fc1b0108f90 2024-04-28T12:49:14.333 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.337+0000 7fc1ad600640 1 -- 172.21.15.106:0/2598487167 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc198004000 con 0x7fc1b0108f90 2024-04-28T12:49:14.334 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.339+0000 7fc1ad600640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc1940a3bb0 0x7fc1940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:14.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.339+0000 7fc1ad600640 1 -- 172.21.15.106:0/2598487167 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc1980d09f0 con 0x7fc1b0108f90 2024-04-28T12:49:14.335 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.339+0000 7fc1afe00640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc1940a3bb0 0x7fc1940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:14.336 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.340+0000 7fc1afe00640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc1940a3bb0 0x7fc1940a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0004ab0 tx=0x7fc1a0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:14.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.344+0000 7fc1ad600640 1 -- 172.21.15.106:0/2598487167 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc19809d190 con 0x7fc1b0108f90 2024-04-28T12:49:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:14 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1066569294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:49:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:14 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1082165588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:49:14.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:14 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/692594625' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-28T12:49:14.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:14 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1066569294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:49:14.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:14 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1082165588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:49:14.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:14 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/692594625' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-28T12:49:14.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.547+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1b002dad0 con 0x7fc1b0108f90 2024-04-28T12:49:14.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.551+0000 7fc1ad600640 1 -- 172.21.15.106:0/2598487167 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7fc19809ca90 con 0x7fc1b0108f90 2024-04-28T12:49:14.547 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:14.547 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc1940a3bb0 msgr2=0x7fc1940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc1940a3bb0 0x7fc1940a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0004ab0 tx=0x7fc1a0004a40 comp rx=0 tx=0).stop 2024-04-28T12:49:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b0108f90 msgr2=0x7fc1b019f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b0108f90 0x7fc1b019f190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e9c0 tx=0x7fc19800ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 shutdown_connections 2024-04-28T12:49:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc1940a3bb0 0x7fc1940a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1b010ab40 0x7fc1b01a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.550 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1b010a180 0x7fc1b019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 --2- 172.21.15.106:0/2598487167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1b0108f90 0x7fc1b019f190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 >> 172.21.15.106:0/2598487167 conn(0x7fc1b0077f40 msgr2=0x7fc1b010d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:14.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 shutdown_connections 2024-04-28T12:49:14.551 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.554+0000 7fc1b7800640 1 -- 172.21.15.106:0/2598487167 wait complete. 2024-04-28T12:49:14.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:48:52.916511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:48:53.528808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:48:57.154563+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:49:14.569 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:49:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.774+0000 7f415f000640 1 -- 172.21.15.106:0/4016344094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 msgr2=0x7f4158104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.774+0000 7f415f000640 1 --2- 172.21.15.106:0/4016344094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 0x7f4158104e90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f414c009900 tx=0x7f414c02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.775+0000 7f415f000640 1 -- 172.21.15.106:0/4016344094 shutdown_connections 2024-04-28T12:49:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.775+0000 7f415f000640 1 --2- 172.21.15.106:0/4016344094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158105c60 0x7f41581060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.775+0000 7f415f000640 1 --2- 172.21.15.106:0/4016344094 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4158106620 0x7f415810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.775+0000 7f415f000640 1 --2- 172.21.15.106:0/4016344094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 0x7f4158104e90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.775+0000 7f415f000640 1 -- 172.21.15.106:0/4016344094 >> 172.21.15.106:0/4016344094 conn(0x7f41580fffe0 msgr2=0x7f4158102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.775+0000 7f415f000640 1 -- 172.21.15.106:0/4016344094 shutdown_connections 2024-04-28T12:49:14.771 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.775+0000 7f415f000640 1 -- 172.21.15.106:0/4016344094 wait complete. 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415f000640 1 Processor -- start 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415f000640 1 -- start start 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415f000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 0x7f415819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415f000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158105c60 0x7f415819b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415f000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4158106620 0x7f41581a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415f000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f41581105f0 con 0x7f4158105c60 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415f000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4158110470 con 0x7f4158104a70 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415f000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4158110770 con 0x7f4158106620 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 0x7f415819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415dc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 0x7f415819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:37884/0 (socket says 172.21.15.106:37884) 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415dc00640 1 -- 172.21.15.106:0/4040126013 learned_addr learned my addr 172.21.15.106:0/4040126013 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.776+0000 7f415d200640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158105c60 0x7f415819b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:14.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.777+0000 7f415e600640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4158106620 0x7f41581a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.777+0000 7f415dc00640 1 -- 172.21.15.106:0/4040126013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4158106620 msgr2=0x7f41581a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.777+0000 7f415dc00640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4158106620 0x7f41581a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.777+0000 7f415dc00640 1 -- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158105c60 msgr2=0x7f415819b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.777+0000 7f415dc00640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158105c60 0x7f415819b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.777+0000 7f415dc00640 1 -- 172.21.15.106:0/4040126013 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41581a2900 con 0x7f4158104a70 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.777+0000 7f415d200640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158105c60 0x7f415819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.777+0000 7f415dc00640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 0x7f415819acc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f414c031500 tx=0x7f414c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.777+0000 7f4156000640 1 -- 172.21.15.106:0/4040126013 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c03f070 con 0x7f4158104a70 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.778+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41581a2b90 con 0x7f4158104a70 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.778+0000 7f4156000640 1 -- 172.21.15.106:0/4040126013 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f414c031c50 con 0x7f4158104a70 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.778+0000 7f4156000640 1 -- 172.21.15.106:0/4040126013 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c035030 con 0x7f4158104a70 2024-04-28T12:49:14.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.778+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41581a30d0 con 0x7f4158104a70 2024-04-28T12:49:14.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.779+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415810bdb0 con 0x7f4158104a70 2024-04-28T12:49:14.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.781+0000 7f4156000640 1 -- 172.21.15.106:0/4040126013 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f414c042070 con 0x7f4158104a70 2024-04-28T12:49:14.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.782+0000 7f4156000640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f41400a3bb0 0x7f41400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:14.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.782+0000 7f4156000640 1 -- 172.21.15.106:0/4040126013 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f414c0f1430 con 0x7f4158104a70 2024-04-28T12:49:14.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.782+0000 7f415d200640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f41400a3bb0 0x7f41400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:14.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.783+0000 7f415d200640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f41400a3bb0 0x7f41400a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f415819c1c0 tx=0x7f4144005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:14.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.784+0000 7f4156000640 1 -- 172.21.15.106:0/4040126013 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f414c0bd520 con 0x7f4158104a70 2024-04-28T12:49:14.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.985+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4158063530 con 0x7f4158104a70 2024-04-28T12:49:14.982 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.986+0000 7f4156000640 1 -- 172.21.15.106:0/4040126013 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f414c0c1820 con 0x7f4158104a70 2024-04-28T12:49:14.982 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.988+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f41400a3bb0 msgr2=0x7f41400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.988+0000 7f415f000640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f41400a3bb0 0x7f41400a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f415819c1c0 tx=0x7f4144005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.988+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 msgr2=0x7f415819acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.988+0000 7f415f000640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 0x7f415819acc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f414c031500 tx=0x7f414c003c10 comp rx=0 tx=0).stop 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.989+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 shutdown_connections 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.989+0000 7f415f000640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f41400a3bb0 0x7f41400a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.989+0000 7f415f000640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4158105c60 0x7f415819b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.989+0000 7f415f000640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4158106620 0x7f41581a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.989+0000 7f415f000640 1 --2- 172.21.15.106:0/4040126013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4158104a70 0x7f415819acc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:14.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.989+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 >> 172.21.15.106:0/4040126013 conn(0x7f41580fffe0 msgr2=0x7f4158108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:14.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.989+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 shutdown_connections 2024-04-28T12:49:14.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:14.989+0000 7f415f000640 1 -- 172.21.15.106:0/4040126013 wait complete. 2024-04-28T12:49:15.003 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:49:15.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.220+0000 7f0ceb200640 1 -- 172.21.15.106:0/4161722026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4109fc0 msgr2=0x7f0ce410a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.220+0000 7f0ceb200640 1 --2- 172.21.15.106:0/4161722026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4109fc0 0x7f0ce410a440 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4009900 tx=0x7f0cd402a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.220+0000 7f0ceb200640 1 -- 172.21.15.106:0/4161722026 shutdown_connections 2024-04-28T12:49:15.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.220+0000 7f0ceb200640 1 --2- 172.21.15.106:0/4161722026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4109fc0 0x7f0ce410a440 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.220+0000 7f0ceb200640 1 --2- 172.21.15.106:0/4161722026 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ce410a980 0x7f0ce41112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.220+0000 7f0ceb200640 1 --2- 172.21.15.106:0/4161722026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce4108e70 0x7f0ce4109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.220+0000 7f0ceb200640 1 -- 172.21.15.106:0/4161722026 >> 172.21.15.106:0/4161722026 conn(0x7f0ce4077e00 msgr2=0x7f0ce407a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:15.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.220+0000 7f0ceb200640 1 -- 172.21.15.106:0/4161722026 shutdown_connections 2024-04-28T12:49:15.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.220+0000 7f0ceb200640 1 -- 172.21.15.106:0/4161722026 wait complete. 2024-04-28T12:49:15.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.222+0000 7f0ceb200640 1 Processor -- start 2024-04-28T12:49:15.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.222+0000 7f0ceb200640 1 -- start start 2024-04-28T12:49:15.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.222+0000 7f0ceb200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4108e70 0x7f0ce419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:15.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.222+0000 7f0ceb200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ce4109fc0 0x7f0ce419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:15.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.222+0000 7f0ceb200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce410a980 0x7f0ce41a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:15.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.222+0000 7f0ceb200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce41149d0 con 0x7f0ce4108e70 2024-04-28T12:49:15.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.222+0000 7f0ceb200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4114850 con 0x7f0ce410a980 2024-04-28T12:49:15.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.222+0000 7f0ceb200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce4114b50 con 0x7f0ce4109fc0 2024-04-28T12:49:15.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4108e70 0x7f0ce419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ce4109fc0 0x7f0ce419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4108e70 0x7f0ce419f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58390/0 (socket says 172.21.15.106:58390) 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ce4109fc0 0x7f0ce419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:34844/0 (socket says 172.21.15.106:34844) 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9e00640 1 -- 172.21.15.106:0/589768996 learned_addr learned my addr 172.21.15.106:0/589768996 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0cea800640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce410a980 0x7f0ce41a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9400640 1 -- 172.21.15.106:0/589768996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce410a980 msgr2=0x7f0ce41a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9400640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce410a980 0x7f0ce41a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9400640 1 -- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4108e70 msgr2=0x7f0ce419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9400640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4108e70 0x7f0ce419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9400640 1 -- 172.21.15.106:0/589768996 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce41a6cf0 con 0x7f0ce4109fc0 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0cea800640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce410a980 0x7f0ce41a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.223+0000 7f0ce9e00640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4108e70 0x7f0ce419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:15.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.224+0000 7f0ce9400640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ce4109fc0 0x7f0ce419f610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006010 tx=0x7f0cd40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:15.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.224+0000 7f0cdea00640 1 -- 172.21.15.106:0/589768996 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd403f030 con 0x7f0ce4109fc0 2024-04-28T12:49:15.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.224+0000 7f0cdea00640 1 -- 172.21.15.106:0/589768996 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cd4031910 con 0x7f0ce4109fc0 2024-04-28T12:49:15.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.224+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce41a6f80 con 0x7f0ce4109fc0 2024-04-28T12:49:15.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.224+0000 7f0cdea00640 1 -- 172.21.15.106:0/589768996 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4035070 con 0x7f0ce4109fc0 2024-04-28T12:49:15.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.224+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce41a73e0 con 0x7f0ce4109fc0 2024-04-28T12:49:15.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.225+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce41101f0 con 0x7f0ce4109fc0 2024-04-28T12:49:15.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.227+0000 7f0cdea00640 1 -- 172.21.15.106:0/589768996 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0cd4042030 con 0x7f0ce4109fc0 2024-04-28T12:49:15.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.229+0000 7f0cdea00640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ccc0a3c30 0x7f0ccc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:15.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.229+0000 7f0cdea00640 1 -- 172.21.15.106:0/589768996 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f0cd40f0b10 con 0x7f0ce4109fc0 2024-04-28T12:49:15.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.229+0000 7f0ce9e00640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ccc0a3c30 0x7f0ccc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:15.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.229+0000 7f0ce9e00640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ccc0a3c30 0x7f0ccc0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80095f0 tx=0x7f0cd8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:15.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.233+0000 7f0cdea00640 1 -- 172.21.15.106:0/589768996 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0cd40bcc00 con 0x7f0ce4109fc0 2024-04-28T12:49:15.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:15 smithi106 ceph-mon[28081]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-28T12:49:15.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:15 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2598487167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:15.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:15 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4040126013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:49:15.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.438+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0ce40635d0 con 0x7f0ce4109fc0 2024-04-28T12:49:15.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.439+0000 7f0cdea00640 1 -- 172.21.15.106:0/589768996 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f0cd40c0f00 con 0x7f0ce4109fc0 2024-04-28T12:49:15.435 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:49:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ccc0a3c30 msgr2=0x7f0ccc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ccc0a3c30 0x7f0ccc0a60f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80095f0 tx=0x7f0cd8005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ce4109fc0 msgr2=0x7f0ce419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ce4109fc0 0x7f0ce419f610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006010 tx=0x7f0cd40042f0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 shutdown_connections 2024-04-28T12:49:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ccc0a3c30 0x7f0ccc0a60f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0ce410a980 0x7f0ce41a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ce4109fc0 0x7f0ce419f610 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 --2- 172.21.15.106:0/589768996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ce4108e70 0x7f0ce419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.442+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 >> 172.21.15.106:0/589768996 conn(0x7f0ce4077e00 msgr2=0x7f0ce410d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.443+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 shutdown_connections 2024-04-28T12:49:15.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.443+0000 7f0ceb200640 1 -- 172.21.15.106:0/589768996 wait complete. 2024-04-28T12:49:15.456 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:49:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:15 smithi171 ceph-mon[63127]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-28T12:49:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:15 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2598487167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:15 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/4040126013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:49:15.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:15 smithi129 ceph-mon[62418]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-28T12:49:15.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:15 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2598487167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:15.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:15 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/4040126013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:49:15.651 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.655+0000 7f9e8f200640 1 -- 172.21.15.106:0/734850641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e90076950 msgr2=0x7f9e90074da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.655+0000 7f9e8f200640 1 --2- 172.21.15.106:0/734850641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e90076950 0x7f9e90074da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009900 tx=0x7f9e8002a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.655+0000 7f9e8f200640 1 -- 172.21.15.106:0/734850641 shutdown_connections 2024-04-28T12:49:15.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.655+0000 7f9e8f200640 1 --2- 172.21.15.106:0/734850641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e9010a5e0 0x7f9e9010ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.655+0000 7f9e8f200640 1 --2- 172.21.15.106:0/734850641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e90075370 0x7f9e900757d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.655+0000 7f9e8f200640 1 --2- 172.21.15.106:0/734850641 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e90076950 0x7f9e90074da0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.655+0000 7f9e8f200640 1 -- 172.21.15.106:0/734850641 >> 172.21.15.106:0/734850641 conn(0x7f9e900fdcc0 msgr2=0x7f9e90100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:15.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.655+0000 7f9e8f200640 1 -- 172.21.15.106:0/734850641 shutdown_connections 2024-04-28T12:49:15.652 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.656+0000 7f9e8f200640 1 -- 172.21.15.106:0/734850641 wait complete. 2024-04-28T12:49:15.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.656+0000 7f9e8f200640 1 Processor -- start 2024-04-28T12:49:15.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8f200640 1 -- start start 2024-04-28T12:49:15.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e90075370 0x7f9e9019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e90076950 0x7f9e9019b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e9010a5e0 0x7f9e901a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9e90110460 con 0x7f9e9010a5e0 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901102e0 con 0x7f9e90075370 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901105e0 con 0x7f9e90076950 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e9010a5e0 0x7f9e901a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e9010a5e0 0x7f9e901a2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58408/0 (socket says 172.21.15.106:58408) 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8e800640 1 -- 172.21.15.106:0/2247069243 learned_addr learned my addr 172.21.15.106:0/2247069243 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8d400640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e90076950 0x7f9e9019b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.657+0000 7f9e8de00640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e90075370 0x7f9e9019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e8e800640 1 -- 172.21.15.106:0/2247069243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e90076950 msgr2=0x7f9e9019b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e8e800640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e90076950 0x7f9e9019b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e8e800640 1 -- 172.21.15.106:0/2247069243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e90075370 msgr2=0x7f9e9019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e8e800640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e90075370 0x7f9e9019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e8e800640 1 -- 172.21.15.106:0/2247069243 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e901a27b0 con 0x7f9e9010a5e0 2024-04-28T12:49:15.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e8e800640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e9010a5e0 0x7f9e901a2140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800efd0 tx=0x7f9e8800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:15.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e86a00640 1 -- 172.21.15.106:0/2247069243 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88003b70 con 0x7f9e9010a5e0 2024-04-28T12:49:15.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e86a00640 1 -- 172.21.15.106:0/2247069243 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e88003d10 con 0x7f9e9010a5e0 2024-04-28T12:49:15.655 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e901a2aa0 con 0x7f9e9010a5e0 2024-04-28T12:49:15.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.658+0000 7f9e86a00640 1 -- 172.21.15.106:0/2247069243 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e880108a0 con 0x7f9e9010a5e0 2024-04-28T12:49:15.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.659+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e901a2ff0 con 0x7f9e9010a5e0 2024-04-28T12:49:15.656 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.660+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e90109360 con 0x7f9e9010a5e0 2024-04-28T12:49:15.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.662+0000 7f9e86a00640 1 -- 172.21.15.106:0/2247069243 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9e88004020 con 0x7f9e9010a5e0 2024-04-28T12:49:15.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.663+0000 7f9e86a00640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9e740a3c30 0x7f9e740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:15.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.663+0000 7f9e86a00640 1 -- 172.21.15.106:0/2247069243 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9e880d0a20 con 0x7f9e9010a5e0 2024-04-28T12:49:15.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.664+0000 7f9e8de00640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9e740a3c30 0x7f9e740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:15.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.664+0000 7f9e8de00640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9e740a3c30 0x7f9e740a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9e80000c00 tx=0x7f9e800314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:15.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.668+0000 7f9e86a00640 1 -- 172.21.15.106:0/2247069243 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9e8809d160 con 0x7f9e9010a5e0 2024-04-28T12:49:15.864 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.868+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e900635d0 con 0x7f9e9010a5e0 2024-04-28T12:49:15.865 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.869+0000 7f9e86a00640 1 -- 172.21.15.106:0/2247069243 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f9e8809ca60 con 0x7f9e9010a5e0 2024-04-28T12:49:15.865 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-28T12:49:15.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9e740a3c30 msgr2=0x7f9e740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9e740a3c30 0x7f9e740a60f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9e80000c00 tx=0x7f9e800314a0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e9010a5e0 msgr2=0x7f9e901a2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:15.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e9010a5e0 0x7f9e901a2140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9e8800efd0 tx=0x7f9e8800c6a0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 shutdown_connections 2024-04-28T12:49:15.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9e740a3c30 0x7f9e740a60f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.868 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e9010a5e0 0x7f9e901a2140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9e90076950 0x7f9e9019b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 --2- 172.21.15.106:0/2247069243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e90075370 0x7f9e9019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:15.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 >> 172.21.15.106:0/2247069243 conn(0x7f9e900fdcc0 msgr2=0x7f9e90106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:15.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 shutdown_connections 2024-04-28T12:49:15.869 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:15.872+0000 7f9e8f200640 1 -- 172.21.15.106:0/2247069243 wait complete. 2024-04-28T12:49:15.887 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:49:16.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.094+0000 7f7332400640 1 -- 172.21.15.106:0/305665591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c104410 msgr2=0x7f732c10aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.094+0000 7f7332400640 1 --2- 172.21.15.106:0/305665591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c104410 0x7f732c10aca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7324009be0 tx=0x7f732402a6c0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.095+0000 7f7332400640 1 -- 172.21.15.106:0/305665591 shutdown_connections 2024-04-28T12:49:16.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.095+0000 7f7332400640 1 --2- 172.21.15.106:0/305665591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c104410 0x7f732c10aca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.095+0000 7f7332400640 1 --2- 172.21.15.106:0/305665591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f732c103a50 0x7f732c103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.095+0000 7f7332400640 1 --2- 172.21.15.106:0/305665591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c102860 0x7f732c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.095+0000 7f7332400640 1 -- 172.21.15.106:0/305665591 >> 172.21.15.106:0/305665591 conn(0x7f732c0fddd0 msgr2=0x7f732c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:16.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.095+0000 7f7332400640 1 -- 172.21.15.106:0/305665591 shutdown_connections 2024-04-28T12:49:16.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.095+0000 7f7332400640 1 -- 172.21.15.106:0/305665591 wait complete. 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.096+0000 7f7332400640 1 Processor -- start 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.096+0000 7f7332400640 1 -- start start 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.096+0000 7f7332400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c102860 0x7f732c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.096+0000 7f7332400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f732c103a50 0x7f732c19b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.096+0000 7f7332400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c104410 0x7f732c1a21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.096+0000 7f7332400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10e370 con 0x7f732c104410 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.096+0000 7f7332400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10e1f0 con 0x7f732c102860 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.096+0000 7f7332400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f732c10e4f0 con 0x7f732c103a50 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7331a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c104410 0x7f732c1a21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7331000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c102860 0x7f732c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7323e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f732c103a50 0x7f732c19b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7331a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c104410 0x7f732c1a21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58428/0 (socket says 172.21.15.106:58428) 2024-04-28T12:49:16.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7331000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c102860 0x7f732c19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:37918/0 (socket says 172.21.15.106:37918) 2024-04-28T12:49:16.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7331a00640 1 -- 172.21.15.106:0/3898621330 learned_addr learned my addr 172.21.15.106:0/3898621330 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:16.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7323e00640 1 -- 172.21.15.106:0/3898621330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c102860 msgr2=0x7f732c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7323e00640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c102860 0x7f732c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7323e00640 1 -- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c104410 msgr2=0x7f732c1a21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7323e00640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c104410 0x7f732c1a21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7323e00640 1 -- 172.21.15.106:0/3898621330 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732c1a2850 con 0x7f732c103a50 2024-04-28T12:49:16.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7331a00640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c104410 0x7f732c1a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:16.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.097+0000 7f7323e00640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f732c103a50 0x7f732c19b180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f731800e9f0 tx=0x7f731800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:16.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.098+0000 7f732a000640 1 -- 172.21.15.106:0/3898621330 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731800ccc0 con 0x7f732c103a50 2024-04-28T12:49:16.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.098+0000 7f732a000640 1 -- 172.21.15.106:0/3898621330 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73180108e0 con 0x7f732c103a50 2024-04-28T12:49:16.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.098+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732c1a2b40 con 0x7f732c103a50 2024-04-28T12:49:16.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.098+0000 7f732a000640 1 -- 172.21.15.106:0/3898621330 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318018880 con 0x7f732c103a50 2024-04-28T12:49:16.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.098+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732c1a30e0 con 0x7f732c103a50 2024-04-28T12:49:16.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.099+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f732c1a32b0 con 0x7f732c103a50 2024-04-28T12:49:16.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.102+0000 7f732a000640 1 -- 172.21.15.106:0/3898621330 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7318010430 con 0x7f732c103a50 2024-04-28T12:49:16.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.103+0000 7f732a000640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f73100a3c30 0x7f73100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:16.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.103+0000 7f732a000640 1 -- 172.21.15.106:0/3898621330 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f73180d09a0 con 0x7f732c103a50 2024-04-28T12:49:16.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.104+0000 7f7331000640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f73100a3c30 0x7f73100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:16.100 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.104+0000 7f7331000640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f73100a3c30 0x7f73100a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f732c19c110 tx=0x7f731c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:16.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.107+0000 7f732a000640 1 -- 172.21.15.106:0/3898621330 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f731809ca90 con 0x7f732c103a50 2024-04-28T12:49:16.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.314+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f732c1a3480 con 0x7f732c103a50 2024-04-28T12:49:16.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.315+0000 7f732a000640 1 -- 172.21.15.106:0/3898621330 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7f73180a0d90 con 0x7f732c103a50 2024-04-28T12:49:16.312 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:49:16.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f73100a3c30 msgr2=0x7f73100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f73100a3c30 0x7f73100a60f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f732c19c110 tx=0x7f731c009260 comp rx=0 tx=0).stop 2024-04-28T12:49:16.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f732c103a50 msgr2=0x7f732c19b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f732c103a50 0x7f732c19b180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f731800e9f0 tx=0x7f731800eeb0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 shutdown_connections 2024-04-28T12:49:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f73100a3c30 0x7f73100a60f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f732c104410 0x7f732c1a21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f732c103a50 0x7f732c19b180 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 --2- 172.21.15.106:0/3898621330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f732c102860 0x7f732c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 >> 172.21.15.106:0/3898621330 conn(0x7f732c0fddd0 msgr2=0x7f732c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 shutdown_connections 2024-04-28T12:49:16.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.318+0000 7f7332400640 1 -- 172.21.15.106:0/3898621330 wait complete. 2024-04-28T12:49:16.332 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T12:49:16.400 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:16 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/589768996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:49:16.400 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:16 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2247069243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:49:16.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:16 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/589768996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:49:16.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:16 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2247069243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:49:16.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:16 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/589768996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:49:16.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:16 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2247069243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:49:16.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.542+0000 7ff7c7200640 1 -- 172.21.15.106:0/280739140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7c8104990 msgr2=0x7ff7c8104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.542+0000 7ff7c7200640 1 --2- 172.21.15.106:0/280739140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7c8104990 0x7ff7c8104db0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009860 tx=0x7ff7b802a340 comp rx=0 tx=0).stop 2024-04-28T12:49:16.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.542+0000 7ff7c7200640 1 -- 172.21.15.106:0/280739140 shutdown_connections 2024-04-28T12:49:16.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.542+0000 7ff7c7200640 1 --2- 172.21.15.106:0/280739140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c8106540 0x7ff7c810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.542+0000 7ff7c7200640 1 --2- 172.21.15.106:0/280739140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8105b80 0x7ff7c8106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.542+0000 7ff7c7200640 1 --2- 172.21.15.106:0/280739140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7c8104990 0x7ff7c8104db0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.542+0000 7ff7c7200640 1 -- 172.21.15.106:0/280739140 >> 172.21.15.106:0/280739140 conn(0x7ff7c80fff00 msgr2=0x7ff7c8102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:16.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.542+0000 7ff7c7200640 1 -- 172.21.15.106:0/280739140 shutdown_connections 2024-04-28T12:49:16.539 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.542+0000 7ff7c7200640 1 -- 172.21.15.106:0/280739140 wait complete. 2024-04-28T12:49:16.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.543+0000 7ff7c7200640 1 Processor -- start 2024-04-28T12:49:16.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c7200640 1 -- start start 2024-04-28T12:49:16.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c7200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7c8104990 0x7ff7c819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:16.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8105b80 0x7ff7c819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:16.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c7200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c8106540 0x7ff7c81a2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:16.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c7200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8110490 con 0x7ff7c8106540 2024-04-28T12:49:16.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c7200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8110310 con 0x7ff7c8104990 2024-04-28T12:49:16.540 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c7200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8110610 con 0x7ff7c8105b80 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c8106540 0x7ff7c81a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c5400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8105b80 0x7ff7c819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c5400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8105b80 0x7ff7c819b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:34900/0 (socket says 172.21.15.106:34900) 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c5400640 1 -- 172.21.15.106:0/1619164089 learned_addr learned my addr 172.21.15.106:0/1619164089 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.544+0000 7ff7c5e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7c8104990 0x7ff7c819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7c5400640 1 -- 172.21.15.106:0/1619164089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7c8104990 msgr2=0x7ff7c819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7c5400640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7c8104990 0x7ff7c819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7c5400640 1 -- 172.21.15.106:0/1619164089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c8106540 msgr2=0x7ff7c81a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7c5400640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c8106540 0x7ff7c81a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7c5400640 1 -- 172.21.15.106:0/1619164089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c81a27e0 con 0x7ff7c8105b80 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7c5400640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8105b80 0x7ff7c819b100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400ea60 tx=0x7ff7b400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:16.542 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7bea00640 1 -- 172.21.15.106:0/1619164089 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b400cde0 con 0x7ff7c8105b80 2024-04-28T12:49:16.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c81a2ad0 con 0x7ff7c8105b80 2024-04-28T12:49:16.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7bea00640 1 -- 172.21.15.106:0/1619164089 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7b40035f0 con 0x7ff7c8105b80 2024-04-28T12:49:16.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.545+0000 7ff7bea00640 1 -- 172.21.15.106:0/1619164089 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4010690 con 0x7ff7c8105b80 2024-04-28T12:49:16.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.546+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c81a3070 con 0x7ff7c8105b80 2024-04-28T12:49:16.543 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.547+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c810bd20 con 0x7ff7c8105b80 2024-04-28T12:49:16.545 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.549+0000 7ff7bea00640 1 -- 172.21.15.106:0/1619164089 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff7b40037a0 con 0x7ff7c8105b80 2024-04-28T12:49:16.546 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.551+0000 7ff7bea00640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff7ac0a3c30 0x7ff7ac0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:16.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.551+0000 7ff7bea00640 1 -- 172.21.15.106:0/1619164089 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff7b40d0870 con 0x7ff7c8105b80 2024-04-28T12:49:16.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.551+0000 7ff7c5e00640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff7ac0a3c30 0x7ff7ac0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:16.547 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.552+0000 7ff7c5e00640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff7ac0a3c30 0x7ff7ac0a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7b802a930 tx=0x7ff7b8030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:16.548 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.552+0000 7ff7bea00640 1 -- 172.21.15.106:0/1619164089 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff7b409cfb0 con 0x7ff7c8105b80 2024-04-28T12:49:16.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.766+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff7c802dad0 con 0x7ff7c8105b80 2024-04-28T12:49:16.763 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.767+0000 7ff7bea00640 1 -- 172.21.15.106:0/1619164089 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+599 (secure 0 0 0) 0x7ff7b409c8b0 con 0x7ff7c8105b80 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6801/2253669851 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6800/2253669851 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/3545890663 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/1369056801 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/938762297 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/1467729753 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6800/102356115 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/2991131145 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6801/102356115 2024-04-29T12:40:21.812705+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/2350345760 2024-04-29T12:40:55.904029+0000 2024-04-28T12:49:16.764 INFO:teuthology.orchestra.run.smithi106.stderr:listed 10 entries 2024-04-28T12:49:16.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff7ac0a3c30 msgr2=0x7ff7ac0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff7ac0a3c30 0x7ff7ac0a60f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7b802a930 tx=0x7ff7b8030470 comp rx=0 tx=0).stop 2024-04-28T12:49:16.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8105b80 msgr2=0x7ff7c819b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:16.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8105b80 0x7ff7c819b100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400ea60 tx=0x7ff7b400ef20 comp rx=0 tx=0).stop 2024-04-28T12:49:16.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 shutdown_connections 2024-04-28T12:49:16.766 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff7ac0a3c30 0x7ff7ac0a60f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7c8106540 0x7ff7c81a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff7c8105b80 0x7ff7c819b100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 --2- 172.21.15.106:0/1619164089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7c8104990 0x7ff7c819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:16.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 >> 172.21.15.106:0/1619164089 conn(0x7ff7c80fff00 msgr2=0x7ff7c8108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:16.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 shutdown_connections 2024-04-28T12:49:16.767 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:16.770+0000 7ff7c7200640 1 -- 172.21.15.106:0/1619164089 wait complete. 2024-04-28T12:49:16.785 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi106.front.sepia.ceph.com... 2024-04-28T12:49:16.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:16.785 DEBUG:teuthology.orchestra.run.smithi106:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-28T12:49:16.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi106.stderr:ceph-fuse[71428]: fuse finished with error 0 and tester_r 0 2024-04-28T12:49:16.875 INFO:teuthology.orchestra.run:waiting for 300 2024-04-28T12:49:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:17 smithi171 ceph-mon[63127]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-28T12:49:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:17 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3898621330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:49:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:17 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1619164089' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-28T12:49:17.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:17 smithi129 ceph-mon[62418]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-28T12:49:17.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:17 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3898621330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:49:17.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:17 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1619164089' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-28T12:49:17.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:17 smithi106 ceph-mon[28081]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-04-28T12:49:17.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:17 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3898621330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:49:17.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:17 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1619164089' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-28T12:49:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:19 smithi171 ceph-mon[63127]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:19.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:19 smithi129 ceph-mon[62418]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:19.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:19 smithi106 ceph-mon[28081]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:21 smithi171 ceph-mon[63127]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:21.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:21 smithi129 ceph-mon[62418]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:21.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:21 smithi106 ceph-mon[28081]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:22.977 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-28T12:49:22.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:49:22.977 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-28T12:49:23.045 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:49:23.245 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:23 smithi171 ceph-mon[63127]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-28T12:49:23.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.257+0000 7fbbefe00640 1 -- 172.21.15.106:0/3689135519 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 msgr2=0x7fbbe810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.257+0000 7fbbefe00640 1 --2- 172.21.15.106:0/3689135519 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 0x7fbbe810a600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009900 tx=0x7fbbd402a410 comp rx=0 tx=0).stop 2024-04-28T12:49:23.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.257+0000 7fbbefe00640 1 -- 172.21.15.106:0/3689135519 shutdown_connections 2024-04-28T12:49:23.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.257+0000 7fbbefe00640 1 --2- 172.21.15.106:0/3689135519 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 0x7fbbe810a600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.257+0000 7fbbefe00640 1 --2- 172.21.15.106:0/3689135519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbe810ab40 0x7fbbe81113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.257+0000 7fbbefe00640 1 --2- 172.21.15.106:0/3689135519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe8108f90 0x7fbbe81093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.257+0000 7fbbefe00640 1 -- 172.21.15.106:0/3689135519 >> 172.21.15.106:0/3689135519 conn(0x7fbbe8077f40 msgr2=0x7fbbe807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:23.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.257+0000 7fbbefe00640 1 -- 172.21.15.106:0/3689135519 shutdown_connections 2024-04-28T12:49:23.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.257+0000 7fbbefe00640 1 -- 172.21.15.106:0/3689135519 wait complete. 2024-04-28T12:49:23.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.258+0000 7fbbefe00640 1 Processor -- start 2024-04-28T12:49:23.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.258+0000 7fbbefe00640 1 -- start start 2024-04-28T12:49:23.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.258+0000 7fbbefe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe8108f90 0x7fbbe819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:23.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.258+0000 7fbbefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 0x7fbbe819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:23.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.258+0000 7fbbefe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbe810ab40 0x7fbbe81a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.258+0000 7fbbefe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8114ad0 con 0x7fbbe810a180 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.258+0000 7fbbefe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8114950 con 0x7fbbe8108f90 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.258+0000 7fbbefe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8114c50 con 0x7fbbe810ab40 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbeca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 0x7fbbe819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbeca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 0x7fbbe819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51406/0 (socket says 172.21.15.106:51406) 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbeca00640 1 -- 172.21.15.106:0/929248043 learned_addr learned my addr 172.21.15.106:0/929248043 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbede00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbe810ab40 0x7fbbe81a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbed400640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe8108f90 0x7fbbe819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbeca00640 1 -- 172.21.15.106:0/929248043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbe810ab40 msgr2=0x7fbbe81a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbeca00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbe810ab40 0x7fbbe81a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbeca00640 1 -- 172.21.15.106:0/929248043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe8108f90 msgr2=0x7fbbe819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbeca00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe8108f90 0x7fbbe819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbeca00640 1 -- 172.21.15.106:0/929248043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe81a6d70 con 0x7fbbe810a180 2024-04-28T12:49:23.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.259+0000 7fbbede00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbe810ab40 0x7fbbe81a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.260+0000 7fbbeca00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 0x7fbbe819f690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4005940 tx=0x7fbbd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.260+0000 7fbbe6000640 1 -- 172.21.15.106:0/929248043 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd403f030 con 0x7fbbe810a180 2024-04-28T12:49:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.260+0000 7fbbe6000640 1 -- 172.21.15.106:0/929248043 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd4003cb0 con 0x7fbbe810a180 2024-04-28T12:49:23.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.260+0000 7fbbe6000640 1 -- 172.21.15.106:0/929248043 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4035070 con 0x7fbbe810a180 2024-04-28T12:49:23.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.260+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe81a7000 con 0x7fbbe810a180 2024-04-28T12:49:23.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.260+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81a74a0 con 0x7fbbe810a180 2024-04-28T12:49:23.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.261+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe81102b0 con 0x7fbbe810a180 2024-04-28T12:49:23.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.263+0000 7fbbe6000640 1 -- 172.21.15.106:0/929248043 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fbbd4042030 con 0x7fbbe810a180 2024-04-28T12:49:23.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.264+0000 7fbbe6000640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbbd00a3bb0 0x7fbbd00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:23.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.265+0000 7fbbe6000640 1 -- 172.21.15.106:0/929248043 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbbd40f0aa0 con 0x7fbbe810a180 2024-04-28T12:49:23.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.265+0000 7fbbed400640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbbd00a3bb0 0x7fbbd00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:23.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.265+0000 7fbbed400640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbbd00a3bb0 0x7fbbd00a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8109fe0 tx=0x7fbbdc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:23.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.269+0000 7fbbe6000640 1 -- 172.21.15.106:0/929248043 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbbd4003e60 con 0x7fbbe810a180 2024-04-28T12:49:23.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.474+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbe8063560 con 0x7fbbe810a180 2024-04-28T12:49:23.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.477+0000 7fbbe6000640 1 -- 172.21.15.106:0/929248043 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7fbbd40bcb90 con 0x7fbbe810a180 2024-04-28T12:49:23.474 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:23.474 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:23.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbbd00a3bb0 msgr2=0x7fbbd00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbbd00a3bb0 0x7fbbd00a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8109fe0 tx=0x7fbbdc005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:23.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 msgr2=0x7fbbe819f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 0x7fbbe819f690 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4005940 tx=0x7fbbd40042f0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 shutdown_connections 2024-04-28T12:49:23.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbbd00a3bb0 0x7fbbd00a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbe810a180 0x7fbbe819f690 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbbe810ab40 0x7fbbe81a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 --2- 172.21.15.106:0/929248043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbe8108f90 0x7fbbe819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 >> 172.21.15.106:0/929248043 conn(0x7fbbe8077f40 msgr2=0x7fbbe810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:23.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 shutdown_connections 2024-04-28T12:49:23.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.480+0000 7fbbefe00640 1 -- 172.21.15.106:0/929248043 wait complete. 2024-04-28T12:49:23.493 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:23 smithi106 ceph-mon[28081]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-28T12:49:23.497 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:49:23.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:23 smithi129 ceph-mon[62418]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-28T12:49:23.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.705+0000 7f455cc00640 1 -- 172.21.15.106:0/352967335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558103b50 msgr2=0x7f4558103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.705+0000 7f455cc00640 1 --2- 172.21.15.106:0/352967335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558103b50 0x7f4558103fd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4540009900 tx=0x7f454002a400 comp rx=0 tx=0).stop 2024-04-28T12:49:23.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.705+0000 7f455cc00640 1 -- 172.21.15.106:0/352967335 shutdown_connections 2024-04-28T12:49:23.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.705+0000 7f455cc00640 1 --2- 172.21.15.106:0/352967335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558103b50 0x7f4558103fd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.705+0000 7f455cc00640 1 --2- 172.21.15.106:0/352967335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4558104510 0x7f455810ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.705+0000 7f455cc00640 1 --2- 172.21.15.106:0/352967335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4558102960 0x7f4558102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.705+0000 7f455cc00640 1 -- 172.21.15.106:0/352967335 >> 172.21.15.106:0/352967335 conn(0x7f45580fdf10 msgr2=0x7f4558100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:23.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.705+0000 7f455cc00640 1 -- 172.21.15.106:0/352967335 shutdown_connections 2024-04-28T12:49:23.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.705+0000 7f455cc00640 1 -- 172.21.15.106:0/352967335 wait complete. 2024-04-28T12:49:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.706+0000 7f455cc00640 1 Processor -- start 2024-04-28T12:49:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.706+0000 7f455cc00640 1 -- start start 2024-04-28T12:49:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.706+0000 7f455cc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4558102960 0x7f455819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.706+0000 7f455cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558103b50 0x7f455819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.706+0000 7f455cc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4558104510 0x7f45581a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f455cc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f455810e490 con 0x7f4558103b50 2024-04-28T12:49:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f455cc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f455810e310 con 0x7f4558104510 2024-04-28T12:49:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f455cc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f455810e610 con 0x7f4558102960 2024-04-28T12:49:23.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f454d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558103b50 0x7f455819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f4555e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4558102960 0x7f455819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f4555e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4558102960 0x7f455819adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:39762/0 (socket says 172.21.15.106:39762) 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f4555e00640 1 -- 172.21.15.106:0/282161975 learned_addr learned my addr 172.21.15.106:0/282161975 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f4556800640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4558104510 0x7f45581a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f4555e00640 1 -- 172.21.15.106:0/282161975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4558104510 msgr2=0x7f45581a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f4555e00640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4558104510 0x7f45581a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f4555e00640 1 -- 172.21.15.106:0/282161975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558103b50 msgr2=0x7f455819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f4555e00640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558103b50 0x7f455819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.707+0000 7f4555e00640 1 -- 172.21.15.106:0/282161975 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45581a2a10 con 0x7f4558102960 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.708+0000 7f4555e00640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4558102960 0x7f455819adf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f454800eb70 tx=0x7f454800ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.708+0000 7f454f400640 1 -- 172.21.15.106:0/282161975 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548003970 con 0x7f4558102960 2024-04-28T12:49:23.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.708+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45581a2d00 con 0x7f4558102960 2024-04-28T12:49:23.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.708+0000 7f454f400640 1 -- 172.21.15.106:0/282161975 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4548004020 con 0x7f4558102960 2024-04-28T12:49:23.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.708+0000 7f454f400640 1 -- 172.21.15.106:0/282161975 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4548010690 con 0x7f4558102960 2024-04-28T12:49:23.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.708+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45581a31d0 con 0x7f4558102960 2024-04-28T12:49:23.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.709+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45581a3400 con 0x7f4558102960 2024-04-28T12:49:23.707 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.711+0000 7f454f400640 1 -- 172.21.15.106:0/282161975 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f45480044d0 con 0x7f4558102960 2024-04-28T12:49:23.708 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.713+0000 7f454f400640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f453c0a3bb0 0x7f453c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:23.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.713+0000 7f454f400640 1 -- 172.21.15.106:0/282161975 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f45480d0dd0 con 0x7f4558102960 2024-04-28T12:49:23.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.713+0000 7f454d400640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f453c0a3bb0 0x7f453c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:23.709 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.714+0000 7f454d400640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f453c0a3bb0 0x7f453c0a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4540000c00 tx=0x7f45400314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:23.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.715+0000 7f454f400640 1 -- 172.21.15.106:0/282161975 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f454809d570 con 0x7f4558102960 2024-04-28T12:49:23.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.919+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45581a35d0 con 0x7f4558102960 2024-04-28T12:49:23.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.921+0000 7f454f400640 1 -- 172.21.15.106:0/282161975 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7f454809ce70 con 0x7f4558102960 2024-04-28T12:49:23.917 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:23.917 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f453c0a3bb0 msgr2=0x7f453c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f453c0a3bb0 0x7f453c0a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4540000c00 tx=0x7f45400314a0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4558102960 msgr2=0x7f455819adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4558102960 0x7f455819adf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f454800eb70 tx=0x7f454800ef70 comp rx=0 tx=0).stop 2024-04-28T12:49:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 shutdown_connections 2024-04-28T12:49:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f453c0a3bb0 0x7f453c0a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558103b50 0x7f455819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4558104510 0x7f45581a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 --2- 172.21.15.106:0/282161975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4558102960 0x7f455819adf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:23.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 >> 172.21.15.106:0/282161975 conn(0x7f45580fdf10 msgr2=0x7f4558109370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:23.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 shutdown_connections 2024-04-28T12:49:23.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:23.924+0000 7f455cc00640 1 -- 172.21.15.106:0/282161975 wait complete. 2024-04-28T12:49:23.940 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-28T12:49:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.153+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1835267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0105d80 msgr2=0x7fc9d0106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:24.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.153+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1835267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0105d80 0x7fc9d0106200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009860 tx=0x7fc9b802a340 comp rx=0 tx=0).stop 2024-04-28T12:49:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.153+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1835267 shutdown_connections 2024-04-28T12:49:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.153+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1835267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9d0106740 0x7fc9d010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.153+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1835267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0105d80 0x7fc9d0106200 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.153+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1835267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc9d0104b90 0x7fc9d0104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.153+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1835267 >> 172.21.15.106:0/1835267 conn(0x7fc9d0100120 msgr2=0x7fc9d0102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.153+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1835267 shutdown_connections 2024-04-28T12:49:24.150 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.153+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1835267 wait complete. 2024-04-28T12:49:24.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.154+0000 7fc9d7a00640 1 Processor -- start 2024-04-28T12:49:24.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.154+0000 7fc9d7a00640 1 -- start start 2024-04-28T12:49:24.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d7a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9d0104b90 0x7fc9d019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:24.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d7a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0105d80 0x7fc9d019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:24.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d7a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc9d0106740 0x7fc9d01a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:24.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d7a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01106a0 con 0x7fc9d0104b90 2024-04-28T12:49:24.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d7a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0110520 con 0x7fc9d0105d80 2024-04-28T12:49:24.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d7a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0110820 con 0x7fc9d0106740 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d5000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9d0104b90 0x7fc9d019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d5000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9d0104b90 0x7fc9d019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51444/0 (socket says 172.21.15.106:51444) 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d5000640 1 -- 172.21.15.106:0/1440451287 learned_addr learned my addr 172.21.15.106:0/1440451287 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9c7e00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0105d80 0x7fc9d019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d5a00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc9d0106740 0x7fc9d01a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d5000640 1 -- 172.21.15.106:0/1440451287 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc9d0106740 msgr2=0x7fc9d01a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d5000640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc9d0106740 0x7fc9d01a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d5000640 1 -- 172.21.15.106:0/1440451287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0105d80 msgr2=0x7fc9d019b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d5000640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0105d80 0x7fc9d019b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9d5000640 1 -- 172.21.15.106:0/1440451287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d01a2a10 con 0x7fc9d0104b90 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.155+0000 7fc9c7e00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0105d80 0x7fc9d019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:24.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.156+0000 7fc9d5000640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9d0104b90 0x7fc9d019adf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e9c0 tx=0x7fc9c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.156+0000 7fc9ce000640 1 -- 172.21.15.106:0/1440451287 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000cd80 con 0x7fc9d0104b90 2024-04-28T12:49:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.156+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d01a2d00 con 0x7fc9d0104b90 2024-04-28T12:49:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.156+0000 7fc9ce000640 1 -- 172.21.15.106:0/1440451287 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9c00035f0 con 0x7fc9d0104b90 2024-04-28T12:49:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.156+0000 7fc9ce000640 1 -- 172.21.15.106:0/1440451287 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0010690 con 0x7fc9d0104b90 2024-04-28T12:49:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.156+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a31c0 con 0x7fc9d0104b90 2024-04-28T12:49:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.157+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d010bf70 con 0x7fc9d0104b90 2024-04-28T12:49:24.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.159+0000 7fc9ce000640 1 -- 172.21.15.106:0/1440451287 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc9c00037a0 con 0x7fc9d0104b90 2024-04-28T12:49:24.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.161+0000 7fc9ce000640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc9b40a3bb0 0x7fc9b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.161+0000 7fc9ce000640 1 -- 172.21.15.106:0/1440451287 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc9c00d09e0 con 0x7fc9d0104b90 2024-04-28T12:49:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.161+0000 7fc9c7e00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc9b40a3bb0 0x7fc9b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:24.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.162+0000 7fc9c7e00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc9b40a3bb0 0x7fc9b40a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8000c00 tx=0x7fc9b8030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:24.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.167+0000 7fc9ce000640 1 -- 172.21.15.106:0/1440451287 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc9c009d120 con 0x7fc9d0104b90 2024-04-28T12:49:24.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:24 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/929248043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:24.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:24 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/282161975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:24.414+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc9d0063600 con 0x7fc9d0104b90 2024-04-28T12:49:24.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:24 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/929248043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:24.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:24 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/282161975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:24.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:24 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/929248043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:24.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:24 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/282161975' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:25.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.171+0000 7fc9ce000640 1 -- 172.21.15.106:0/1440451287 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fc9c009ca20 con 0x7fc9d0104b90 2024-04-28T12:49:25.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.175+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc9b40a3bb0 msgr2=0x7fc9b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:25.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc9b40a3bb0 0x7fc9b40a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8000c00 tx=0x7fc9b8030470 comp rx=0 tx=0).stop 2024-04-28T12:49:25.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9d0104b90 msgr2=0x7fc9d019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:25.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9d0104b90 0x7fc9d019adf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e9c0 tx=0x7fc9c000ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:25.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 shutdown_connections 2024-04-28T12:49:25.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc9b40a3bb0 0x7fc9b40a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc9d0106740 0x7fc9d01a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0105d80 0x7fc9d019b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 --2- 172.21.15.106:0/1440451287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9d0104b90 0x7fc9d019adf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 >> 172.21.15.106:0/1440451287 conn(0x7fc9d0100120 msgr2=0x7fc9d0109140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:25.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 shutdown_connections 2024-04-28T12:49:25.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.176+0000 7fc9d7a00640 1 -- 172.21.15.106:0/1440451287 wait complete. 2024-04-28T12:49:25.197 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-28T12:49:25.197 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-28T12:49:25.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.406+0000 7f818b200640 1 -- 172.21.15.106:0/2379702492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f818c103390 msgr2=0x7f818c105780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:25.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.406+0000 7f818b200640 1 --2- 172.21.15.106:0/2379702492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f818c103390 0x7f818c105780 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f817c009900 tx=0x7f817c02a410 comp rx=0 tx=0).stop 2024-04-28T12:49:25.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.406+0000 7f818b200640 1 -- 172.21.15.106:0/2379702492 shutdown_connections 2024-04-28T12:49:25.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.406+0000 7f818b200640 1 --2- 172.21.15.106:0/2379702492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f818c1085f0 0x7f818c10a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.406+0000 7f818b200640 1 --2- 172.21.15.106:0/2379702492 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c105cc0 0x7f818c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.406+0000 7f818b200640 1 --2- 172.21.15.106:0/2379702492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f818c103390 0x7f818c105780 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.406+0000 7f818b200640 1 -- 172.21.15.106:0/2379702492 >> 172.21.15.106:0/2379702492 conn(0x7f818c0fcce0 msgr2=0x7f818c0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:25.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.406+0000 7f818b200640 1 -- 172.21.15.106:0/2379702492 shutdown_connections 2024-04-28T12:49:25.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.406+0000 7f818b200640 1 -- 172.21.15.106:0/2379702492 wait complete. 2024-04-28T12:49:25.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.407+0000 7f818b200640 1 Processor -- start 2024-04-28T12:49:25.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.407+0000 7f818b200640 1 -- start start 2024-04-28T12:49:25.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f818b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f818c103390 0x7f818c102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:25.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f818b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f818c105cc0 0x7f818c101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:25.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f818b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c1085f0 0x7f818c101880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f818b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f818c10e400 con 0x7f818c105cc0 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f818b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f818c10e280 con 0x7f818c103390 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f818b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f818c10e580 con 0x7f818c1085f0 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8189e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f818c103390 0x7f818c102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8181400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f818c105cc0 0x7f818c101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8189e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f818c103390 0x7f818c102d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:53598/0 (socket says 172.21.15.106:53598) 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8189e00640 1 -- 172.21.15.106:0/1374239780 learned_addr learned my addr 172.21.15.106:0/1374239780 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f818a800640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c1085f0 0x7f818c101880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8181400640 1 -- 172.21.15.106:0/1374239780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c1085f0 msgr2=0x7f818c101880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8181400640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c1085f0 0x7f818c101880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8181400640 1 -- 172.21.15.106:0/1374239780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f818c103390 msgr2=0x7f818c102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8181400640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f818c103390 0x7f818c102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8181400640 1 -- 172.21.15.106:0/1374239780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c102110 con 0x7f818c105cc0 2024-04-28T12:49:25.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.408+0000 7f8189e00640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f818c103390 0x7f818c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:25.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.409+0000 7f8181400640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f818c105cc0 0x7f818c101340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f817400e9c0 tx=0x7f817400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:25.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.409+0000 7f8183400640 1 -- 172.21.15.106:0/1374239780 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817400ccc0 con 0x7f818c105cc0 2024-04-28T12:49:25.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.409+0000 7f8183400640 1 -- 172.21.15.106:0/1374239780 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8174005020 con 0x7f818c105cc0 2024-04-28T12:49:25.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.409+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818c1a2c50 con 0x7f818c105cc0 2024-04-28T12:49:25.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.409+0000 7f8183400640 1 -- 172.21.15.106:0/1374239780 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174004a00 con 0x7f818c105cc0 2024-04-28T12:49:25.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.409+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f818c1a31a0 con 0x7f818c105cc0 2024-04-28T12:49:25.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.410+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818c0fe7d0 con 0x7f818c105cc0 2024-04-28T12:49:25.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.412+0000 7f8183400640 1 -- 172.21.15.106:0/1374239780 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f81740051d0 con 0x7f818c105cc0 2024-04-28T12:49:25.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.413+0000 7f8183400640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81700a3c30 0x7f81700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:25.409 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.414+0000 7f8183400640 1 -- 172.21.15.106:0/1374239780 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f81740d05e0 con 0x7f818c105cc0 2024-04-28T12:49:25.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.414+0000 7f8189e00640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81700a3c30 0x7f81700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:25.410 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.414+0000 7f8189e00640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81700a3c30 0x7f81700a60f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f817c0096c0 tx=0x7f817c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:25.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.418+0000 7f8183400640 1 -- 172.21.15.106:0/1374239780 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f817409cd20 con 0x7f818c105cc0 2024-04-28T12:49:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:25 smithi171 ceph-mon[63127]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:25 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1440451287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-28T12:49:25.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:25 smithi129 ceph-mon[62418]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:25.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:25 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1440451287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-28T12:49:25.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:25 smithi106 ceph-mon[28081]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:25.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:25 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1440451287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-28T12:49:25.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.661+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f818c063530 con 0x7f818c105cc0 2024-04-28T12:49:25.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.960+0000 7f8183400640 1 -- 172.21.15.106:0/1374239780 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f817409c620 con 0x7f818c105cc0 2024-04-28T12:49:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81700a3c30 msgr2=0x7f81700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81700a3c30 0x7f81700a60f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f817c0096c0 tx=0x7f817c005a20 comp rx=0 tx=0).stop 2024-04-28T12:49:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f818c105cc0 msgr2=0x7f818c101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f818c105cc0 0x7f818c101340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f817400e9c0 tx=0x7f817400ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 shutdown_connections 2024-04-28T12:49:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81700a3c30 0x7f81700a60f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f818c105cc0 0x7f818c101340 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c1085f0 0x7f818c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.962 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 --2- 172.21.15.106:0/1374239780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f818c103390 0x7f818c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:25.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 >> 172.21.15.106:0/1374239780 conn(0x7f818c0fcce0 msgr2=0x7f818c103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:25.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 shutdown_connections 2024-04-28T12:49:25.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:25.966+0000 7f818b200640 1 -- 172.21.15.106:0/1374239780 wait complete. 2024-04-28T12:49:25.988 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-28T12:49:26.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.178+0000 7f8532200640 1 -- 172.21.15.106:0/910711049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c105c60 msgr2=0x7f852c1060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:26.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.178+0000 7f8532200640 1 --2- 172.21.15.106:0/910711049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c105c60 0x7f852c1060e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8518009900 tx=0x7f851802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.178+0000 7f8532200640 1 -- 172.21.15.106:0/910711049 shutdown_connections 2024-04-28T12:49:26.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.178+0000 7f8532200640 1 --2- 172.21.15.106:0/910711049 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c106620 0x7f852c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.178+0000 7f8532200640 1 --2- 172.21.15.106:0/910711049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c105c60 0x7f852c1060e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.178+0000 7f8532200640 1 --2- 172.21.15.106:0/910711049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f852c104a70 0x7f852c104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.178+0000 7f8532200640 1 -- 172.21.15.106:0/910711049 >> 172.21.15.106:0/910711049 conn(0x7f852c0fffe0 msgr2=0x7f852c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.178+0000 7f8532200640 1 -- 172.21.15.106:0/910711049 shutdown_connections 2024-04-28T12:49:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.178+0000 7f8532200640 1 -- 172.21.15.106:0/910711049 wait complete. 2024-04-28T12:49:26.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.179+0000 7f8532200640 1 Processor -- start 2024-04-28T12:49:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.179+0000 7f8532200640 1 -- start start 2024-04-28T12:49:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f8532200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c104a70 0x7f852c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f8532200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c105c60 0x7f852c19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f8532200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f852c106620 0x7f852c1a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f8532200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1105f0 con 0x7f852c106620 2024-04-28T12:49:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f8532200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f852c110470 con 0x7f852c104a70 2024-04-28T12:49:26.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f8532200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f852c110770 con 0x7f852c105c60 2024-04-28T12:49:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f852be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c105c60 0x7f852c19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f852be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c105c60 0x7f852c19b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:39806/0 (socket says 172.21.15.106:39806) 2024-04-28T12:49:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f852be00640 1 -- 172.21.15.106:0/2303368972 learned_addr learned my addr 172.21.15.106:0/2303368972 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f8530e00640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c104a70 0x7f852c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.180+0000 7f8531800640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f852c106620 0x7f852c1a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.181+0000 7f852be00640 1 -- 172.21.15.106:0/2303368972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c104a70 msgr2=0x7f852c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.181+0000 7f852be00640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c104a70 0x7f852c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.181+0000 7f852be00640 1 -- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f852c106620 msgr2=0x7f852c1a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:26.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.181+0000 7f852be00640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f852c106620 0x7f852c1a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.181+0000 7f852be00640 1 -- 172.21.15.106:0/2303368972 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c1a2890 con 0x7f852c105c60 2024-04-28T12:49:26.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.181+0000 7f8530e00640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c104a70 0x7f852c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:26.178 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.181+0000 7f852be00640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c105c60 0x7f852c19b1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8518005ab0 tx=0x7f8518003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:26.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.184+0000 7f8529600640 1 -- 172.21.15.106:0/2303368972 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851803f070 con 0x7f852c105c60 2024-04-28T12:49:26.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.184+0000 7f8529600640 1 -- 172.21.15.106:0/2303368972 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8518031c50 con 0x7f852c105c60 2024-04-28T12:49:26.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.184+0000 7f8529600640 1 -- 172.21.15.106:0/2303368972 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518035030 con 0x7f852c105c60 2024-04-28T12:49:26.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.184+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852c1a2b20 con 0x7f852c105c60 2024-04-28T12:49:26.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.185+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852c1a3060 con 0x7f852c105c60 2024-04-28T12:49:26.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.186+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c10bd90 con 0x7f852c105c60 2024-04-28T12:49:26.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.188+0000 7f8529600640 1 -- 172.21.15.106:0/2303368972 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8518042070 con 0x7f852c105c60 2024-04-28T12:49:26.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.189+0000 7f8529600640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f85140a3c30 0x7f85140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:26.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.189+0000 7f8529600640 1 -- 172.21.15.106:0/2303368972 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f85180f1440 con 0x7f852c105c60 2024-04-28T12:49:26.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.190+0000 7f8530e00640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f85140a3c30 0x7f85140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:26.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.190+0000 7f8530e00640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f85140a3c30 0x7f85140a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f852c19c150 tx=0x7f852000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:26.188 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.192+0000 7f8529600640 1 -- 172.21.15.106:0/2303368972 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f85180bd530 con 0x7f852c105c60 2024-04-28T12:49:26.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:26 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1440451287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-28T12:49:26.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:26 smithi106 ceph-mon[28081]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-28T12:49:26.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:26 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1374239780' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-28T12:49:26.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:26 smithi106 ceph-mon[28081]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-28T12:49:26.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:26 smithi106 ceph-mon[28081]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-28T12:49:26.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:26 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1374239780' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-28T12:49:26.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:26 smithi106 ceph-mon[28081]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-28T12:49:26.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:26 smithi106 ceph-mon[28081]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-28T12:49:26.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.436+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f852c02dad0 con 0x7f852c105c60 2024-04-28T12:49:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:26 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1440451287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-28T12:49:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:26 smithi171 ceph-mon[63127]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-28T12:49:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:26 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1374239780' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-28T12:49:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:26 smithi171 ceph-mon[63127]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-28T12:49:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:26 smithi171 ceph-mon[63127]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-28T12:49:26.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:26 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1374239780' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-28T12:49:26.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:26 smithi171 ceph-mon[63127]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-28T12:49:26.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:26 smithi171 ceph-mon[63127]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-28T12:49:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:26 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1440451287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-28T12:49:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:26 smithi129 ceph-mon[62418]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-28T12:49:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:26 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1374239780' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-28T12:49:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:26 smithi129 ceph-mon[62418]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-28T12:49:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:26 smithi129 ceph-mon[62418]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-28T12:49:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:26 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1374239780' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-28T12:49:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:26 smithi129 ceph-mon[62418]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-28T12:49:26.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:26 smithi129 ceph-mon[62418]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-28T12:49:26.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.975+0000 7f8529600640 1 -- 172.21.15.106:0/2303368972 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f85180c1830 con 0x7f852c105c60 2024-04-28T12:49:26.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f85140a3c30 msgr2=0x7f85140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f85140a3c30 0x7f85140a60f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f852c19c150 tx=0x7f852000b390 comp rx=0 tx=0).stop 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c105c60 msgr2=0x7f852c19b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c105c60 0x7f852c19b1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8518005ab0 tx=0x7f8518003b90 comp rx=0 tx=0).stop 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 shutdown_connections 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f85140a3c30 0x7f85140a60f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f852c106620 0x7f852c1a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f852c105c60 0x7f852c19b1c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 --2- 172.21.15.106:0/2303368972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f852c104a70 0x7f852c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.981+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 >> 172.21.15.106:0/2303368972 conn(0x7f852c0fffe0 msgr2=0x7f852c108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.982+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 shutdown_connections 2024-04-28T12:49:26.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:26.982+0000 7f8532200640 1 -- 172.21.15.106:0/2303368972 wait complete. 2024-04-28T12:49:27.006 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-28T12:49:27.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.192+0000 7f5981a00640 1 -- 172.21.15.106:0/3473469041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c101280 msgr2=0x7f597c1016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:27.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.192+0000 7f5981a00640 1 --2- 172.21.15.106:0/3473469041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c101280 0x7f597c1016a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f596c009900 tx=0x7f596c02a480 comp rx=0 tx=0).stop 2024-04-28T12:49:27.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.193+0000 7f5981a00640 1 -- 172.21.15.106:0/3473469041 shutdown_connections 2024-04-28T12:49:27.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.193+0000 7f5981a00640 1 --2- 172.21.15.106:0/3473469041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f597c101be0 0x7f597c102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:27.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.193+0000 7f5981a00640 1 --2- 172.21.15.106:0/3473469041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f597c102580 0x7f597c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:27.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.193+0000 7f5981a00640 1 --2- 172.21.15.106:0/3473469041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c101280 0x7f597c1016a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:27.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.193+0000 7f5981a00640 1 -- 172.21.15.106:0/3473469041 >> 172.21.15.106:0/3473469041 conn(0x7f597c0fce20 msgr2=0x7f597c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:27.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.193+0000 7f5981a00640 1 -- 172.21.15.106:0/3473469041 shutdown_connections 2024-04-28T12:49:27.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.193+0000 7f5981a00640 1 -- 172.21.15.106:0/3473469041 wait complete. 2024-04-28T12:49:27.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.194+0000 7f5981a00640 1 Processor -- start 2024-04-28T12:49:27.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.194+0000 7f5981a00640 1 -- start start 2024-04-28T12:49:27.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.194+0000 7f5981a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f597c101280 0x7f597c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:27.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.194+0000 7f5981a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f597c101be0 0x7f597c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:27.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.194+0000 7f5981a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c102580 0x7f597c1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:27.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.194+0000 7f5981a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f597c1106e0 con 0x7f597c101be0 2024-04-28T12:49:27.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.194+0000 7f5981a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f597c110560 con 0x7f597c102580 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.194+0000 7f5981a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f597c110860 con 0x7f597c101280 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c102580 0x7f597c1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c102580 0x7f597c1a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:53630/0 (socket says 172.21.15.106:53630) 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597b400640 1 -- 172.21.15.106:0/3819440788 learned_addr learned my addr 172.21.15.106:0/3819440788 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597aa00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f597c101280 0x7f597c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f5972000640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f597c101be0 0x7f597c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597b400640 1 -- 172.21.15.106:0/3819440788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f597c101280 msgr2=0x7f597c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597b400640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f597c101280 0x7f597c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597b400640 1 -- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f597c101be0 msgr2=0x7f597c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597b400640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f597c101be0 0x7f597c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597b400640 1 -- 172.21.15.106:0/3819440788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c1a29d0 con 0x7f597c102580 2024-04-28T12:49:27.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.195+0000 7f597b400640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c102580 0x7f597c1a2330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5974009fd0 tx=0x7f597400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:27.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.196+0000 7f5973e00640 1 -- 172.21.15.106:0/3819440788 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974017070 con 0x7f597c102580 2024-04-28T12:49:27.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.196+0000 7f5973e00640 1 -- 172.21.15.106:0/3819440788 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5974003e20 con 0x7f597c102580 2024-04-28T12:49:27.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.196+0000 7f5973e00640 1 -- 172.21.15.106:0/3819440788 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974010460 con 0x7f597c102580 2024-04-28T12:49:27.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.196+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c1a2cc0 con 0x7f597c102580 2024-04-28T12:49:27.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.196+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c1a3190 con 0x7f597c102580 2024-04-28T12:49:27.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.197+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c10bfd0 con 0x7f597c102580 2024-04-28T12:49:27.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.199+0000 7f5973e00640 1 -- 172.21.15.106:0/3819440788 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5974003970 con 0x7f597c102580 2024-04-28T12:49:27.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.200+0000 7f5973e00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f59600a3c30 0x7f59600a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:27.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.200+0000 7f5973e00640 1 -- 172.21.15.106:0/3819440788 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f59740d0380 con 0x7f597c102580 2024-04-28T12:49:27.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.200+0000 7f597aa00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f59600a3c30 0x7f59600a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:27.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.201+0000 7f597aa00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f59600a3c30 0x7f59600a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f596c031500 tx=0x7f596c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:27.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.204+0000 7f5973e00640 1 -- 172.21.15.106:0/3819440788 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f597409c4b0 con 0x7f597c102580 2024-04-28T12:49:27.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:27 smithi106 ceph-mon[28081]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:27.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:27 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2303368972' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-28T12:49:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:27 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-28T12:49:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:27 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-28T12:49:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:27 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] up:active 2024-04-28T12:49:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:27 smithi106 ceph-mon[28081]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-28T12:49:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:27 smithi106 ceph-mon[28081]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-28T12:49:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:27 smithi106 ceph-mon[28081]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-04-28T12:49:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:27 smithi106 ceph-mon[28081]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-28T12:49:27.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.446+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f597c063530 con 0x7f597c102580 2024-04-28T12:49:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:27 smithi171 ceph-mon[63127]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:27 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2303368972' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-28T12:49:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:27 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-28T12:49:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:27 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-28T12:49:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:27 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] up:active 2024-04-28T12:49:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:27 smithi171 ceph-mon[63127]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-28T12:49:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:27 smithi171 ceph-mon[63127]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-28T12:49:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:27 smithi171 ceph-mon[63127]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-04-28T12:49:27.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:27 smithi171 ceph-mon[63127]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-28T12:49:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:27 smithi129 ceph-mon[62418]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-28T12:49:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:27 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2303368972' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-28T12:49:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:27 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-28T12:49:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:27 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-28T12:49:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:27 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] up:active 2024-04-28T12:49:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:27 smithi129 ceph-mon[62418]: daemon mds.g assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-28T12:49:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:27 smithi129 ceph-mon[62418]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-28T12:49:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:27 smithi129 ceph-mon[62418]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:creating} 9 up:standby 2024-04-28T12:49:27.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:27 smithi129 ceph-mon[62418]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-28T12:49:27.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.987+0000 7f5973e00640 1 -- 172.21.15.106:0/3819440788 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f59740a07b0 con 0x7f597c102580 2024-04-28T12:49:27.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f59600a3c30 msgr2=0x7f59600a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:27.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f59600a3c30 0x7f59600a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f596c031500 tx=0x7f596c005a20 comp rx=0 tx=0).stop 2024-04-28T12:49:27.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c102580 msgr2=0x7f597c1a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:27.986 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c102580 0x7f597c1a2330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5974009fd0 tx=0x7f597400b260 comp rx=0 tx=0).stop 2024-04-28T12:49:27.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 shutdown_connections 2024-04-28T12:49:27.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f59600a3c30 0x7f59600a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:27.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f597c101be0 0x7f597c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:27.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f597c102580 0x7f597c1a2330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:27.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 --2- 172.21.15.106:0/3819440788 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f597c101280 0x7f597c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:27.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 >> 172.21.15.106:0/3819440788 conn(0x7f597c0fce20 msgr2=0x7f597c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:27.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 shutdown_connections 2024-04-28T12:49:27.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:27.990+0000 7f5981a00640 1 -- 172.21.15.106:0/3819440788 wait complete. 2024-04-28T12:49:28.012 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-28T12:49:28.012 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-28T12:49:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.197+0000 7f5006a00640 1 -- 172.21.15.106:0/3903026115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000101770 msgr2=0x7f5000101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.197+0000 7f5006a00640 1 --2- 172.21.15.106:0/3903026115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000101770 0x7f5000101b90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009900 tx=0x7f4ff402a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.198+0000 7f5006a00640 1 -- 172.21.15.106:0/3903026115 shutdown_connections 2024-04-28T12:49:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.198+0000 7f5006a00640 1 --2- 172.21.15.106:0/3903026115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5000103320 0x7f5000109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.198+0000 7f5006a00640 1 --2- 172.21.15.106:0/3903026115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5000102960 0x7f5000102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.198+0000 7f5006a00640 1 --2- 172.21.15.106:0/3903026115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000101770 0x7f5000101b90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.198+0000 7f5006a00640 1 -- 172.21.15.106:0/3903026115 >> 172.21.15.106:0/3903026115 conn(0x7f50000fcce0 msgr2=0x7f50000ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:28.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.198+0000 7f5006a00640 1 -- 172.21.15.106:0/3903026115 shutdown_connections 2024-04-28T12:49:28.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.198+0000 7f5006a00640 1 -- 172.21.15.106:0/3903026115 wait complete. 2024-04-28T12:49:28.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.199+0000 7f5006a00640 1 Processor -- start 2024-04-28T12:49:28.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.199+0000 7f5006a00640 1 -- start start 2024-04-28T12:49:28.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.199+0000 7f5006a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5000101770 0x7f500019ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:28.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.199+0000 7f5006a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5000102960 0x7f500019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:28.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.199+0000 7f5006a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000103320 0x7f50001a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:28.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.199+0000 7f5006a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f500010d290 con 0x7f5000101770 2024-04-28T12:49:28.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.199+0000 7f5006a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f500010d110 con 0x7f5000103320 2024-04-28T12:49:28.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.199+0000 7f5006a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f500010d410 con 0x7f5000102960 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5005600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5000101770 0x7f500019ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5006000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000103320 0x7f50001a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5005600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5000101770 0x7f500019ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51494/0 (socket says 172.21.15.106:51494) 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5005600640 1 -- 172.21.15.106:0/1514045845 learned_addr learned my addr 172.21.15.106:0/1514045845 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5004c00640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5000102960 0x7f500019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5005600640 1 -- 172.21.15.106:0/1514045845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5000102960 msgr2=0x7f500019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5005600640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5000102960 0x7f500019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5005600640 1 -- 172.21.15.106:0/1514045845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000103320 msgr2=0x7f50001a2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5005600640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000103320 0x7f50001a2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5005600640 1 -- 172.21.15.106:0/1514045845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50001a2910 con 0x7f5000101770 2024-04-28T12:49:28.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.200+0000 7f5005600640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5000101770 0x7f500019ad00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4006010 tx=0x7f4ff40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.201+0000 7f4ffe000640 1 -- 172.21.15.106:0/1514045845 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff403f030 con 0x7f5000101770 2024-04-28T12:49:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.201+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50001a2ba0 con 0x7f5000101770 2024-04-28T12:49:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.201+0000 7f4ffe000640 1 -- 172.21.15.106:0/1514045845 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff4031910 con 0x7f5000101770 2024-04-28T12:49:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.201+0000 7f4ffe000640 1 -- 172.21.15.106:0/1514045845 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4035070 con 0x7f5000101770 2024-04-28T12:49:28.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.201+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50001a30e0 con 0x7f5000101770 2024-04-28T12:49:28.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.202+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5000068de0 con 0x7f5000101770 2024-04-28T12:49:28.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.204+0000 7f4ffe000640 1 -- 172.21.15.106:0/1514045845 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4ff4042030 con 0x7f5000101770 2024-04-28T12:49:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.205+0000 7f4ffe000640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4fe80a3bb0 0x7f4fe80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.205+0000 7f4ffe000640 1 -- 172.21.15.106:0/1514045845 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4ff40f0c00 con 0x7f5000101770 2024-04-28T12:49:28.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.206+0000 7f5004c00640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4fe80a3bb0 0x7f4fe80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:28.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.206+0000 7f5004c00640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4fe80a3bb0 0x7f4fe80a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009bd0 tx=0x7f4fec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:28.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.210+0000 7f4ffe000640 1 -- 172.21.15.106:0/1514045845 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4ff40bccf0 con 0x7f5000101770 2024-04-28T12:49:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3819440788' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-28T12:49:28.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-28T12:49:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-28T12:49:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] up:active 2024-04-28T12:49:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-mon[28081]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-28T12:49:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-mon[28081]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-28T12:49:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-mon[28081]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:creating} 8 up:standby 2024-04-28T12:49:28.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-mon[28081]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-04-28T12:49:28.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:28.437+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f50000fef30 con 0x7f4fe80a3bb0 2024-04-28T12:49:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:28 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3819440788' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-28T12:49:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:28 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-28T12:49:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:28 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-28T12:49:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:28 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] up:active 2024-04-28T12:49:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:28 smithi171 ceph-mon[63127]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-28T12:49:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:28 smithi171 ceph-mon[63127]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-28T12:49:28.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:28 smithi171 ceph-mon[63127]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:creating} 8 up:standby 2024-04-28T12:49:28.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:28 smithi171 ceph-mon[63127]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-04-28T12:49:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:28 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3819440788' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-28T12:49:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:28 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-28T12:49:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:28 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-28T12:49:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:28 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] up:active 2024-04-28T12:49:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:28 smithi129 ceph-mon[62418]: daemon mds.i assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-28T12:49:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:28 smithi129 ceph-mon[62418]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=g=up:active} 9 up:standby 2024-04-28T12:49:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:28 smithi129 ceph-mon[62418]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:creating} 8 up:standby 2024-04-28T12:49:28.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:28 smithi129 ceph-mon[62418]: daemon mds.i is now active in filesystem cephfs as rank 3 2024-04-28T12:49:28.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:49:28.444+0000 7f2b22000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T12:49:28.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:49:28.444+0000 7f2b22000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T12:49:28.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:49:28.444+0000 7f2b22000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T12:49:28.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:49:28.444+0000 7f2b22000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T12:49:28.825 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 12:49:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T12:49:28.444+0000 7f2b22000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-28T12:49:29.047 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.050+0000 7f4ffe000640 1 -- 172.21.15.106:0/1514045845 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f50000fef30 con 0x7f4fe80a3bb0 2024-04-28T12:49:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4fe80a3bb0 msgr2=0x7f4fe80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4fe80a3bb0 0x7f4fe80a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009bd0 tx=0x7f4fec005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5000101770 msgr2=0x7f500019ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5000101770 0x7f500019ad00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4006010 tx=0x7f4ff40042f0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 shutdown_connections 2024-04-28T12:49:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4fe80a3bb0 0x7f4fe80a6070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000103320 0x7f50001a2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5000102960 0x7f500019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 --2- 172.21.15.106:0/1514045845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5000101770 0x7f500019ad00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 >> 172.21.15.106:0/1514045845 conn(0x7f50000fcce0 msgr2=0x7f50000fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 shutdown_connections 2024-04-28T12:49:29.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.054+0000 7f5006a00640 1 -- 172.21.15.106:0/1514045845 wait complete. 2024-04-28T12:49:29.070 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-28T12:49:29.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.272+0000 7f6f4b200640 1 -- 172.21.15.106:0/1766919107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c103a50 msgr2=0x7f6f4c103ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.272+0000 7f6f4b200640 1 --2- 172.21.15.106:0/1766919107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c103a50 0x7f6f4c103ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009900 tx=0x7f6f3402a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.272+0000 7f6f4b200640 1 -- 172.21.15.106:0/1766919107 shutdown_connections 2024-04-28T12:49:29.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.272+0000 7f6f4b200640 1 --2- 172.21.15.106:0/1766919107 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f4c104410 0x7f6f4c10aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.272+0000 7f6f4b200640 1 --2- 172.21.15.106:0/1766919107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c103a50 0x7f6f4c103ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.272+0000 7f6f4b200640 1 --2- 172.21.15.106:0/1766919107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f4c102860 0x7f6f4c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.272+0000 7f6f4b200640 1 -- 172.21.15.106:0/1766919107 >> 172.21.15.106:0/1766919107 conn(0x7f6f4c0fddd0 msgr2=0x7f6f4c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:29.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.272+0000 7f6f4b200640 1 -- 172.21.15.106:0/1766919107 shutdown_connections 2024-04-28T12:49:29.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.272+0000 7f6f4b200640 1 -- 172.21.15.106:0/1766919107 wait complete. 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.273+0000 7f6f4b200640 1 Processor -- start 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.273+0000 7f6f4b200640 1 -- start start 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f4b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c102860 0x7f6f4c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f4b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f4c103a50 0x7f6f4c198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f4b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f4c104410 0x7f6f4c1a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f4b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10e390 con 0x7f6f4c104410 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f4b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10e210 con 0x7f6f4c102860 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f4b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c10e510 con 0x7f6f4c103a50 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f49e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c102860 0x7f6f4c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f4a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f4c104410 0x7f6f4c1a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f49e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c102860 0x7f6f4c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:60408/0 (socket says 172.21.15.106:60408) 2024-04-28T12:49:29.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f49e00640 1 -- 172.21.15.106:0/771523832 learned_addr learned my addr 172.21.15.106:0/771523832 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f43e00640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f4c103a50 0x7f6f4c198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f49e00640 1 -- 172.21.15.106:0/771523832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f4c103a50 msgr2=0x7f6f4c198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f49e00640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f4c103a50 0x7f6f4c198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f49e00640 1 -- 172.21.15.106:0/771523832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f4c104410 msgr2=0x7f6f4c1a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f49e00640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f4c104410 0x7f6f4c1a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f49e00640 1 -- 172.21.15.106:0/771523832 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c1a0690 con 0x7f6f4c102860 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.274+0000 7f6f43e00640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f4c103a50 0x7f6f4c198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.275+0000 7f6f49e00640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c102860 0x7f6f4c198a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9c0 tx=0x7f6f3c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.275+0000 7f6f42a00640 1 -- 172.21.15.106:0/771523832 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00cde0 con 0x7f6f4c102860 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.275+0000 7f6f42a00640 1 -- 172.21.15.106:0/771523832 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f3c0108e0 con 0x7f6f4c102860 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.275+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c1a0980 con 0x7f6f4c102860 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.275+0000 7f6f42a00640 1 -- 172.21.15.106:0/771523832 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c018960 con 0x7f6f4c102860 2024-04-28T12:49:29.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.275+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c1a0ed0 con 0x7f6f4c102860 2024-04-28T12:49:29.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.276+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c109c00 con 0x7f6f4c102860 2024-04-28T12:49:29.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.278+0000 7f6f42a00640 1 -- 172.21.15.106:0/771523832 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6f3c01f020 con 0x7f6f4c102860 2024-04-28T12:49:29.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.279+0000 7f6f42a00640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6f300a3bb0 0x7f6f300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:29.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.279+0000 7f6f42a00640 1 -- 172.21.15.106:0/771523832 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f6f3c0d09c0 con 0x7f6f4c102860 2024-04-28T12:49:29.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.279+0000 7f6f43e00640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6f300a3bb0 0x7f6f300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:29.276 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.280+0000 7f6f43e00640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6f300a3bb0 0x7f6f300a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c199f80 tx=0x7f6f340314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:29.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.284+0000 7f6f42a00640 1 -- 172.21.15.106:0/771523832 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f6f3c054060 con 0x7f6f4c102860 2024-04-28T12:49:29.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:29 smithi106 ceph-mon[28081]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-04-28T12:49:29.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:29 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] up:active 2024-04-28T12:49:29.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:29 smithi106 ceph-mon[28081]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-28T12:49:29.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:29 smithi106 ceph-mon[28081]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-28T12:49:29.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:29 smithi106 ceph-mon[28081]: Cluster is now healthy 2024-04-28T12:49:29.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:29 smithi106 ceph-mon[28081]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:active} 8 up:standby 2024-04-28T12:49:29.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:29 smithi106 ceph-mon[28081]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-28T12:49:29.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:29 smithi106 ceph-mon[28081]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-28T12:49:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:29 smithi171 ceph-mon[63127]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-04-28T12:49:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:29 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] up:active 2024-04-28T12:49:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:29 smithi171 ceph-mon[63127]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-28T12:49:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:29 smithi171 ceph-mon[63127]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-28T12:49:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:29 smithi171 ceph-mon[63127]: Cluster is now healthy 2024-04-28T12:49:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:29 smithi171 ceph-mon[63127]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:active} 8 up:standby 2024-04-28T12:49:29.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:29 smithi171 ceph-mon[63127]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-28T12:49:29.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:29 smithi171 ceph-mon[63127]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-28T12:49:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.515+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6f4c062d70 con 0x7f6f300a3bb0 2024-04-28T12:49:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:29 smithi129 ceph-mon[62418]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 1 op/s 2024-04-28T12:49:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:29 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] up:active 2024-04-28T12:49:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:29 smithi129 ceph-mon[62418]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-28T12:49:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:29 smithi129 ceph-mon[62418]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-28T12:49:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:29 smithi129 ceph-mon[62418]: Cluster is now healthy 2024-04-28T12:49:29.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:29 smithi129 ceph-mon[62418]: fsmap cephfs:4 {0=b=up:active,1=a=up:active,2=g=up:active,3=i=up:active} 8 up:standby 2024-04-28T12:49:29.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:29 smithi129 ceph-mon[62418]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-28T12:49:29.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:29 smithi129 ceph-mon[62418]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-28T12:49:29.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.559+0000 7f6f42a00640 1 -- 172.21.15.106:0/771523832 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f4c062d70 con 0x7f6f300a3bb0 2024-04-28T12:49:29.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6f300a3bb0 msgr2=0x7f6f300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6f300a3bb0 0x7f6f300a6070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c199f80 tx=0x7f6f340314a0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c102860 msgr2=0x7f6f4c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c102860 0x7f6f4c198a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9c0 tx=0x7f6f3c00ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:29.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 shutdown_connections 2024-04-28T12:49:29.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f6f300a3bb0 0x7f6f300a6070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f4c104410 0x7f6f4c1a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f4c103a50 0x7f6f4c198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 --2- 172.21.15.106:0/771523832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f4c102860 0x7f6f4c198a70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 >> 172.21.15.106:0/771523832 conn(0x7f6f4c0fddd0 msgr2=0x7f6f4c106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:29.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 shutdown_connections 2024-04-28T12:49:29.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.563+0000 7f6f4b200640 1 -- 172.21.15.106:0/771523832 wait complete. 2024-04-28T12:49:29.579 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-28T12:49:29.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.789+0000 7ff20f200640 1 -- 172.21.15.106:0/1302034943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 msgr2=0x7ff210077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.789+0000 7ff20f200640 1 --2- 172.21.15.106:0/1302034943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 0x7ff210077710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff200009900 tx=0x7ff20002a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.789+0000 7ff20f200640 1 -- 172.21.15.106:0/1302034943 shutdown_connections 2024-04-28T12:49:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.789+0000 7ff20f200640 1 --2- 172.21.15.106:0/1302034943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff210078ed0 0x7ff210079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.789+0000 7ff20f200640 1 --2- 172.21.15.106:0/1302034943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff210078530 0x7ff210078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.789+0000 7ff20f200640 1 --2- 172.21.15.106:0/1302034943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 0x7ff210077710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.789+0000 7ff20f200640 1 -- 172.21.15.106:0/1302034943 >> 172.21.15.106:0/1302034943 conn(0x7ff2100fffe0 msgr2=0x7ff210102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.789+0000 7ff20f200640 1 -- 172.21.15.106:0/1302034943 shutdown_connections 2024-04-28T12:49:29.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.789+0000 7ff20f200640 1 -- 172.21.15.106:0/1302034943 wait complete. 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.790+0000 7ff20f200640 1 Processor -- start 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20f200640 1 -- start start 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 0x7ff21019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff210078530 0x7ff21019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff210078ed0 0x7ff2101a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101149d0 con 0x7ff210078ed0 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff210114850 con 0x7ff2100772f0 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff210114b50 con 0x7ff210078530 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 0x7ff21019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 0x7ff21019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:60426/0 (socket says 172.21.15.106:60426) 2024-04-28T12:49:29.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20de00640 1 -- 172.21.15.106:0/576704265 learned_addr learned my addr 172.21.15.106:0/576704265 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20e800640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff210078ed0 0x7ff2101a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.791+0000 7ff20d400640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff210078530 0x7ff21019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.792+0000 7ff20de00640 1 -- 172.21.15.106:0/576704265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff210078530 msgr2=0x7ff21019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.792+0000 7ff20de00640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff210078530 0x7ff21019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.792+0000 7ff20de00640 1 -- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff210078ed0 msgr2=0x7ff2101a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.792+0000 7ff20de00640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff210078ed0 0x7ff2101a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.792+0000 7ff20de00640 1 -- 172.21.15.106:0/576704265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2101a6d10 con 0x7ff2100772f0 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.792+0000 7ff20de00640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 0x7ff21019f0f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff200005ab0 tx=0x7ff200003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.792+0000 7ff206a00640 1 -- 172.21.15.106:0/576704265 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20003f070 con 0x7ff2100772f0 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.792+0000 7ff206a00640 1 -- 172.21.15.106:0/576704265 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff200031c50 con 0x7ff2100772f0 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.792+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2101a6fa0 con 0x7ff2100772f0 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.793+0000 7ff206a00640 1 -- 172.21.15.106:0/576704265 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200035030 con 0x7ff2100772f0 2024-04-28T12:49:29.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.793+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2101a7400 con 0x7ff2100772f0 2024-04-28T12:49:29.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.794+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2101113c0 con 0x7ff2100772f0 2024-04-28T12:49:29.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.796+0000 7ff206a00640 1 -- 172.21.15.106:0/576704265 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff200042070 con 0x7ff2100772f0 2024-04-28T12:49:29.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.797+0000 7ff206a00640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff1f40a3c30 0x7ff1f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:29.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.797+0000 7ff206a00640 1 -- 172.21.15.106:0/576704265 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff2000f0cc0 con 0x7ff2100772f0 2024-04-28T12:49:29.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.797+0000 7ff20d400640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff1f40a3c30 0x7ff1f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:29.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.798+0000 7ff20d400640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff1f40a3c30 0x7ff1f40a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2101a0600 tx=0x7ff1fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:29.797 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:29.802+0000 7ff206a00640 1 -- 172.21.15.106:0/576704265 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2000f7080 con 0x7ff2100772f0 2024-04-28T12:49:30.043 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.047+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff210062d70 con 0x7ff1f40a3c30 2024-04-28T12:49:30.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.053+0000 7ff206a00640 1 -- 172.21.15.106:0/576704265 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff210062d70 con 0x7ff1f40a3c30 2024-04-28T12:49:30.049 INFO:teuthology.orchestra.run.smithi106.stdout:/volumes/qa/sv_0/30d68533-4bdb-4df3-8f60-3eead385064e 2024-04-28T12:49:30.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff1f40a3c30 msgr2=0x7ff1f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff1f40a3c30 0x7ff1f40a60f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2101a0600 tx=0x7ff1fc005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 msgr2=0x7ff21019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 0x7ff21019f0f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff200005ab0 tx=0x7ff200003b90 comp rx=0 tx=0).stop 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 shutdown_connections 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff1f40a3c30 0x7ff1f40a60f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff210078ed0 0x7ff2101a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff210078530 0x7ff21019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 --2- 172.21.15.106:0/576704265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2100772f0 0x7ff21019f0f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.055+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 >> 172.21.15.106:0/576704265 conn(0x7ff2100fffe0 msgr2=0x7ff210103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.056+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 shutdown_connections 2024-04-28T12:49:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.056+0000 7ff20f200640 1 -- 172.21.15.106:0/576704265 wait complete. 2024-04-28T12:49:30.069 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-28T12:49:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.258+0000 7fe728c00640 1 -- 172.21.15.106:0/3914199429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe724103b60 msgr2=0x7fe724103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.258+0000 7fe728c00640 1 --2- 172.21.15.106:0/3914199429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe724103b60 0x7fe724103fe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009ca0 tx=0x7fe70c02a8c0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.258+0000 7fe728c00640 1 -- 172.21.15.106:0/3914199429 shutdown_connections 2024-04-28T12:49:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.258+0000 7fe728c00640 1 --2- 172.21.15.106:0/3914199429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe724104520 0x7fe72410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.258+0000 7fe728c00640 1 --2- 172.21.15.106:0/3914199429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe724103b60 0x7fe724103fe0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.258+0000 7fe728c00640 1 --2- 172.21.15.106:0/3914199429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe724102970 0x7fe724102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.258+0000 7fe728c00640 1 -- 172.21.15.106:0/3914199429 >> 172.21.15.106:0/3914199429 conn(0x7fe7240fdee0 msgr2=0x7fe724100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.258+0000 7fe728c00640 1 -- 172.21.15.106:0/3914199429 shutdown_connections 2024-04-28T12:49:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.258+0000 7fe728c00640 1 -- 172.21.15.106:0/3914199429 wait complete. 2024-04-28T12:49:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.260+0000 7fe728c00640 1 Processor -- start 2024-04-28T12:49:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.260+0000 7fe728c00640 1 -- start start 2024-04-28T12:49:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.260+0000 7fe728c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe724102970 0x7fe72419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe728c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe724103b60 0x7fe72419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe728c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe724104520 0x7fe7241a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe728c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe72410e4a0 con 0x7fe724102970 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe728c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe72410e320 con 0x7fe724104520 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe728c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe72410e620 con 0x7fe724103b60 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe721e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe724102970 0x7fe72419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe722800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe724104520 0x7fe7241a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe721e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe724102970 0x7fe72419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43212/0 (socket says 172.21.15.106:43212) 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe721e00640 1 -- 172.21.15.106:0/1278772675 learned_addr learned my addr 172.21.15.106:0/1278772675 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe721400640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe724103b60 0x7fe72419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe722800640 1 -- 172.21.15.106:0/1278772675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe724103b60 msgr2=0x7fe72419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe722800640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe724103b60 0x7fe72419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe722800640 1 -- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe724102970 msgr2=0x7fe72419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe722800640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe724102970 0x7fe72419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe722800640 1 -- 172.21.15.106:0/1278772675 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7241a2a10 con 0x7fe724104520 2024-04-28T12:49:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.261+0000 7fe721e00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe724102970 0x7fe72419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.262+0000 7fe722800640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe724104520 0x7fe7241a23a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e970 tx=0x7fe71400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.262+0000 7fe71aa00640 1 -- 172.21.15.106:0/1278772675 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71400ccc0 con 0x7fe724104520 2024-04-28T12:49:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.262+0000 7fe71aa00640 1 -- 172.21.15.106:0/1278772675 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe714005020 con 0x7fe724104520 2024-04-28T12:49:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.262+0000 7fe71aa00640 1 -- 172.21.15.106:0/1278772675 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714004a00 con 0x7fe724104520 2024-04-28T12:49:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.262+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7241a2d00 con 0x7fe724104520 2024-04-28T12:49:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.262+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7241a31a0 con 0x7fe724104520 2024-04-28T12:49:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.263+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7241a3430 con 0x7fe724104520 2024-04-28T12:49:30.260 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.265+0000 7fe71aa00640 1 -- 172.21.15.106:0/1278772675 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe714004ba0 con 0x7fe724104520 2024-04-28T12:49:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.266+0000 7fe71aa00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7080a3c30 0x7fe7080a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.266+0000 7fe71aa00640 1 -- 172.21.15.106:0/1278772675 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe7140d0510 con 0x7fe724104520 2024-04-28T12:49:30.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.266+0000 7fe721e00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7080a3c30 0x7fe7080a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:30.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.267+0000 7fe721e00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7080a3c30 0x7fe7080a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0058c0 tx=0x7fe71c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:30.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.270+0000 7fe71aa00640 1 -- 172.21.15.106:0/1278772675 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe71409cc50 con 0x7fe724104520 2024-04-28T12:49:30.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:30 smithi106 ceph-mon[28081]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:30.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:30 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] up:active 2024-04-28T12:49:30.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:30 smithi106 ceph-mon[28081]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-28T12:49:30.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:30 smithi106 ceph-mon[28081]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-28T12:49:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.498+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe724076280 con 0x7fe7080a3c30 2024-04-28T12:49:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:30 smithi171 ceph-mon[63127]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:30 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] up:active 2024-04-28T12:49:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:30 smithi171 ceph-mon[63127]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-28T12:49:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:30 smithi171 ceph-mon[63127]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-28T12:49:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:30 smithi129 ceph-mon[62418]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:30 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] up:active 2024-04-28T12:49:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:30 smithi129 ceph-mon[62418]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-28T12:49:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:30 smithi129 ceph-mon[62418]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-28T12:49:30.513 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:49:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: 2024-04-28T12:49:30.076+0000 7f952dbf8700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-28T12:49:30.513 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:49:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: 2024-04-28T12:49:30.076+0000 7f952dbf8700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-28T12:49:30.514 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:49:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: 2024-04-28T12:49:30.076+0000 7f952dbf8700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-28T12:49:30.514 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 12:49:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: 2024-04-28T12:49:30.076+0000 7f952dbf8700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-28T12:49:30.531 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.536+0000 7fe71aa00640 1 -- 172.21.15.106:0/1278772675 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe724076280 con 0x7fe7080a3c30 2024-04-28T12:49:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7080a3c30 msgr2=0x7fe7080a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7080a3c30 0x7fe7080a60f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0058c0 tx=0x7fe71c005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe724104520 msgr2=0x7fe7241a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.534 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe724104520 0x7fe7241a23a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe71400e970 tx=0x7fe71400ee30 comp rx=0 tx=0).stop 2024-04-28T12:49:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 shutdown_connections 2024-04-28T12:49:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7080a3c30 0x7fe7080a60f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe724104520 0x7fe7241a23a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe724103b60 0x7fe72419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 --2- 172.21.15.106:0/1278772675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe724102970 0x7fe72419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 >> 172.21.15.106:0/1278772675 conn(0x7fe7240fdee0 msgr2=0x7fe724074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 shutdown_connections 2024-04-28T12:49:30.535 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.539+0000 7fe728c00640 1 -- 172.21.15.106:0/1278772675 wait complete. 2024-04-28T12:49:30.552 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-28T12:49:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.757+0000 7f004e200640 1 -- 172.21.15.106:0/861893810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 msgr2=0x7f0048101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.757+0000 7f004e200640 1 --2- 172.21.15.106:0/861893810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 0x7f0048101b70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0038009900 tx=0x7f003802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.758+0000 7f004e200640 1 -- 172.21.15.106:0/861893810 shutdown_connections 2024-04-28T12:49:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.758+0000 7f004e200640 1 --2- 172.21.15.106:0/861893810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0048103300 0x7f0048109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.758+0000 7f004e200640 1 --2- 172.21.15.106:0/861893810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048102940 0x7f0048102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.758+0000 7f004e200640 1 --2- 172.21.15.106:0/861893810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 0x7f0048101b70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.758+0000 7f004e200640 1 -- 172.21.15.106:0/861893810 >> 172.21.15.106:0/861893810 conn(0x7f00480fcce0 msgr2=0x7f00480ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.758+0000 7f004e200640 1 -- 172.21.15.106:0/861893810 shutdown_connections 2024-04-28T12:49:30.753 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.758+0000 7f004e200640 1 -- 172.21.15.106:0/861893810 wait complete. 2024-04-28T12:49:30.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004e200640 1 Processor -- start 2024-04-28T12:49:30.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004e200640 1 -- start start 2024-04-28T12:49:30.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004e200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 0x7f004819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:30.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004e200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0048102940 0x7f004819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:30.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004e200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048103300 0x7f00481a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004e200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f004810d270 con 0x7f0048102940 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004e200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f004810d0f0 con 0x7f0048101750 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004e200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f004810d3f0 con 0x7f0048103300 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 0x7f004819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004ce00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 0x7f004819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:60458/0 (socket says 172.21.15.106:60458) 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004ce00640 1 -- 172.21.15.106:0/3059048038 learned_addr learned my addr 172.21.15.106:0/3059048038 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f003fe00640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0048102940 0x7f004819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.759+0000 7f004d800640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048103300 0x7f00481a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f004ce00640 1 -- 172.21.15.106:0/3059048038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048103300 msgr2=0x7f00481a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f004ce00640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048103300 0x7f00481a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f004ce00640 1 -- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0048102940 msgr2=0x7f004819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f004ce00640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0048102940 0x7f004819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:30.756 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f004ce00640 1 -- 172.21.15.106:0/3059048038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00481a2960 con 0x7f0048101750 2024-04-28T12:49:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f004ce00640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 0x7f004819ad50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0038006010 tx=0x7f00380042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f0046000640 1 -- 172.21.15.106:0/3059048038 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003803f030 con 0x7f0048101750 2024-04-28T12:49:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f0046000640 1 -- 172.21.15.106:0/3059048038 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0038031910 con 0x7f0048101750 2024-04-28T12:49:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00481a2bf0 con 0x7f0048101750 2024-04-28T12:49:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f0046000640 1 -- 172.21.15.106:0/3059048038 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038035070 con 0x7f0048101750 2024-04-28T12:49:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.760+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00481a30e0 con 0x7f0048101750 2024-04-28T12:49:30.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.761+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00481a32b0 con 0x7f0048101750 2024-04-28T12:49:30.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.763+0000 7f0046000640 1 -- 172.21.15.106:0/3059048038 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0038042030 con 0x7f0048101750 2024-04-28T12:49:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.765+0000 7f0046000640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f002c0a3bb0 0x7f002c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.765+0000 7f0046000640 1 -- 172.21.15.106:0/3059048038 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f00380f10c0 con 0x7f0048101750 2024-04-28T12:49:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.765+0000 7f003fe00640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f002c0a3bb0 0x7f002c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:30.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.766+0000 7f003fe00640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f002c0a3bb0 0x7f002c0a6070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0030009bd0 tx=0x7f0030005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:30.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:30.767+0000 7f0046000640 1 -- 172.21.15.106:0/3059048038 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f00380bd800 con 0x7f0048101750 2024-04-28T12:49:31.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.005+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f00480fda90 con 0x7f002c0a3bb0 2024-04-28T12:49:31.007 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.012+0000 7f0046000640 1 -- 172.21.15.106:0/3059048038 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f00480fda90 con 0x7f002c0a3bb0 2024-04-28T12:49:31.007 INFO:teuthology.orchestra.run.smithi106.stdout:/volumes/qa/sv_1/be9407c1-65bc-4ae7-81c7-e36ca57f5e10 2024-04-28T12:49:31.009 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f002c0a3bb0 msgr2=0x7f002c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f002c0a3bb0 0x7f002c0a6070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0030009bd0 tx=0x7f0030005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 msgr2=0x7f004819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 0x7f004819ad50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0038006010 tx=0x7f00380042f0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 shutdown_connections 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f002c0a3bb0 0x7f002c0a6070 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0048102940 0x7f004819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0048103300 0x7f00481a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 --2- 172.21.15.106:0/3059048038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0048101750 0x7f004819ad50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 >> 172.21.15.106:0/3059048038 conn(0x7f00480fcce0 msgr2=0x7f00480fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 shutdown_connections 2024-04-28T12:49:31.010 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.014+0000 7f004e200640 1 -- 172.21.15.106:0/3059048038 wait complete. 2024-04-28T12:49:31.029 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:49:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.223+0000 7f72a3200640 1 -- 172.21.15.106:0/2582077984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72a4074720 msgr2=0x7f72a410b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.223+0000 7f72a3200640 1 --2- 172.21.15.106:0/2582077984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72a4074720 0x7f72a410b940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7290009900 tx=0x7f729002a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.223+0000 7f72a3200640 1 -- 172.21.15.106:0/2582077984 shutdown_connections 2024-04-28T12:49:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.223+0000 7f72a3200640 1 --2- 172.21.15.106:0/2582077984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a410e9c0 0x7f72a4119870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.223+0000 7f72a3200640 1 --2- 172.21.15.106:0/2582077984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a410bf10 0x7f72a410e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.223+0000 7f72a3200640 1 --2- 172.21.15.106:0/2582077984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72a4074720 0x7f72a410b940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.223+0000 7f72a3200640 1 -- 172.21.15.106:0/2582077984 >> 172.21.15.106:0/2582077984 conn(0x7f72a406f5a0 msgr2=0x7f72a40719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.223+0000 7f72a3200640 1 -- 172.21.15.106:0/2582077984 shutdown_connections 2024-04-28T12:49:31.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.223+0000 7f72a3200640 1 -- 172.21.15.106:0/2582077984 wait complete. 2024-04-28T12:49:31.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.224+0000 7f72a3200640 1 Processor -- start 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.224+0000 7f72a3200640 1 -- start start 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a3200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a4074720 0x7f72a41a7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72a410bf10 0x7f72a41a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a3200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a410e9c0 0x7f72a41aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a3200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f72a411d0c0 con 0x7f72a4074720 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a3200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72a411cf40 con 0x7f72a410e9c0 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f72a411d240 con 0x7f72a410bf10 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a1e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a4074720 0x7f72a41a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a1e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a4074720 0x7f72a41a7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43254/0 (socket says 172.21.15.106:43254) 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a1e00640 1 -- 172.21.15.106:0/22764538 learned_addr learned my addr 172.21.15.106:0/22764538 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:31.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a1400640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72a410bf10 0x7f72a41a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.225+0000 7f72a2800640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a410e9c0 0x7f72a41aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.226+0000 7f72a1400640 1 -- 172.21.15.106:0/22764538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a410e9c0 msgr2=0x7f72a41aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.226+0000 7f72a1400640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a410e9c0 0x7f72a41aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.226+0000 7f72a1400640 1 -- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a4074720 msgr2=0x7f72a41a7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.226+0000 7f72a1400640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a4074720 0x7f72a41a7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.226+0000 7f72a1400640 1 -- 172.21.15.106:0/22764538 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a41af3b0 con 0x7f72a410bf10 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.226+0000 7f72a1e00640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a4074720 0x7f72a41a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.226+0000 7f72a1400640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72a410bf10 0x7f72a41a7cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f729400e9f0 tx=0x7f729400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.226+0000 7f729aa00640 1 -- 172.21.15.106:0/22764538 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729400ccc0 con 0x7f72a410bf10 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.226+0000 7f729aa00640 1 -- 172.21.15.106:0/22764538 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72940108e0 con 0x7f72a410bf10 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.227+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a41af6a0 con 0x7f72a410bf10 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.227+0000 7f729aa00640 1 -- 172.21.15.106:0/22764538 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294018880 con 0x7f72a410bf10 2024-04-28T12:49:31.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.227+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a41afcc0 con 0x7f72a410bf10 2024-04-28T12:49:31.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.228+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a4071090 con 0x7f72a410bf10 2024-04-28T12:49:31.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.230+0000 7f729aa00640 1 -- 172.21.15.106:0/22764538 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7294010430 con 0x7f72a410bf10 2024-04-28T12:49:31.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.231+0000 7f729aa00640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f72880a3cc0 0x7f72880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:31.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.231+0000 7f729aa00640 1 -- 172.21.15.106:0/22764538 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f72940d0aa0 con 0x7f72a410bf10 2024-04-28T12:49:31.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.231+0000 7f72a1e00640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f72880a3cc0 0x7f72880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:31.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.232+0000 7f72a1e00640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f72880a3cc0 0x7f72880a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7290009230 tx=0x7f72900058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:31.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.236+0000 7f729aa00640 1 -- 172.21.15.106:0/22764538 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f729409ca30 con 0x7f72a410bf10 2024-04-28T12:49:31.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:31 smithi129 ceph-mon[62418]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:31.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:31 smithi129 ceph-mon[62418]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:31.263 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:31 smithi129 ceph-mon[62418]: pgmap v336: 129 pgs: 129 active+clean; 585 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 5 op/s 2024-04-28T12:49:31.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:31 smithi106 ceph-mon[28081]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:31.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:31 smithi106 ceph-mon[28081]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:31.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:31 smithi106 ceph-mon[28081]: pgmap v336: 129 pgs: 129 active+clean; 585 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 5 op/s 2024-04-28T12:49:31.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.480+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72a402dad0 con 0x7f72a410bf10 2024-04-28T12:49:31.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.481+0000 7f729aa00640 1 -- 172.21.15.106:0/22764538 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11953 (secure 0 0 0) 0x7f72940a0d30 con 0x7f72a410bf10 2024-04-28T12:49:31.477 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:31.477 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:49:30.023676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24548,"mds_3":24538,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24548,24520,24578,24538,24566]},"id":1}]} 2024-04-28T12:49:31.477 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-28T12:49:31.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f72880a3cc0 msgr2=0x7f72880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f72880a3cc0 0x7f72880a6180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7290009230 tx=0x7f72900058d0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72a410bf10 msgr2=0x7f72a41a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72a410bf10 0x7f72a41a7cd0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f729400e9f0 tx=0x7f729400eeb0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 shutdown_connections 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f72880a3cc0 0x7f72880a6180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a410e9c0 0x7f72a41aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72a410bf10 0x7f72a41a7cd0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 --2- 172.21.15.106:0/22764538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a4074720 0x7f72a41a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 >> 172.21.15.106:0/22764538 conn(0x7f72a406f5a0 msgr2=0x7f72a4109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.484+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 shutdown_connections 2024-04-28T12:49:31.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.485+0000 7f72a3200640 1 -- 172.21.15.106:0/22764538 wait complete. 2024-04-28T12:49:31.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:31 smithi171 ceph-mon[63127]: from='client.24830 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:31.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:31 smithi171 ceph-mon[63127]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:31.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:31 smithi171 ceph-mon[63127]: pgmap v336: 129 pgs: 129 active+clean; 585 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 5 op/s 2024-04-28T12:49:31.499 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:49:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.700+0000 7fb407000640 1 -- 172.21.15.106:0/2526340783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb40010ab40 msgr2=0x7fb4001113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.700+0000 7fb407000640 1 --2- 172.21.15.106:0/2526340783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb40010ab40 0x7fb4001113d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009ca0 tx=0x7fb3f802a8c0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.701+0000 7fb407000640 1 -- 172.21.15.106:0/2526340783 shutdown_connections 2024-04-28T12:49:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.701+0000 7fb407000640 1 --2- 172.21.15.106:0/2526340783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb40010ab40 0x7fb4001113d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.701+0000 7fb407000640 1 --2- 172.21.15.106:0/2526340783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb40010a180 0x7fb40010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.701+0000 7fb407000640 1 --2- 172.21.15.106:0/2526340783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb400108f90 0x7fb4001093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.701+0000 7fb407000640 1 -- 172.21.15.106:0/2526340783 >> 172.21.15.106:0/2526340783 conn(0x7fb400077f40 msgr2=0x7fb40007a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:31.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.701+0000 7fb407000640 1 -- 172.21.15.106:0/2526340783 shutdown_connections 2024-04-28T12:49:31.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.701+0000 7fb407000640 1 -- 172.21.15.106:0/2526340783 wait complete. 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.702+0000 7fb407000640 1 Processor -- start 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.702+0000 7fb407000640 1 -- start start 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.702+0000 7fb407000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb400108f90 0x7fb40019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.702+0000 7fb407000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40010a180 0x7fb40019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.702+0000 7fb407000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb40010ab40 0x7fb4001a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.702+0000 7fb407000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb400114b20 con 0x7fb40010ab40 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.702+0000 7fb407000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4001149a0 con 0x7fb40010a180 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.702+0000 7fb407000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb400114ca0 con 0x7fb400108f90 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb405000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb40010ab40 0x7fb4001a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:31.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3ffe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb400108f90 0x7fb40019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3f7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40010a180 0x7fb40019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3ffe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb400108f90 0x7fb40019f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:50402/0 (socket says 172.21.15.106:50402) 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3ffe00640 1 -- 172.21.15.106:0/2991303226 learned_addr learned my addr 172.21.15.106:0/2991303226 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3f7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40010a180 0x7fb40019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:60478/0 (socket says 172.21.15.106:60478) 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3ffe00640 1 -- 172.21.15.106:0/2991303226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40010a180 msgr2=0x7fb40019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3ffe00640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40010a180 0x7fb40019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3ffe00640 1 -- 172.21.15.106:0/2991303226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb40010ab40 msgr2=0x7fb4001a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3ffe00640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb40010ab40 0x7fb4001a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3ffe00640 1 -- 172.21.15.106:0/2991303226 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4001a6d40 con 0x7fb400108f90 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3f7e00640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40010a180 0x7fb40019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.703+0000 7fb3ffe00640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb400108f90 0x7fb40019f120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000e9c0 tx=0x7fb3f000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.704+0000 7fb3fd600640 1 -- 172.21.15.106:0/2991303226 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f000cde0 con 0x7fb400108f90 2024-04-28T12:49:31.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.704+0000 7fb3fd600640 1 -- 172.21.15.106:0/2991303226 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f00108e0 con 0x7fb400108f90 2024-04-28T12:49:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.704+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4001a6fd0 con 0x7fb400108f90 2024-04-28T12:49:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.704+0000 7fb3fd600640 1 -- 172.21.15.106:0/2991303226 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00188d0 con 0x7fb400108f90 2024-04-28T12:49:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.704+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4001a7490 con 0x7fb400108f90 2024-04-28T12:49:31.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.705+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4001102b0 con 0x7fb400108f90 2024-04-28T12:49:31.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.707+0000 7fb3fd600640 1 -- 172.21.15.106:0/2991303226 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb3f001f020 con 0x7fb400108f90 2024-04-28T12:49:31.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.708+0000 7fb3fd600640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb3e40a3c40 0x7fb3e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:31.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.708+0000 7fb3fd600640 1 -- 172.21.15.106:0/2991303226 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb3f00d0b90 con 0x7fb400108f90 2024-04-28T12:49:31.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.709+0000 7fb3f7e00640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb3e40a3c40 0x7fb3e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:31.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.709+0000 7fb3f7e00640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb3e40a3c40 0x7fb3e40a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb4001a0630 tx=0x7fb3e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:31.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.710+0000 7fb3fd600640 1 -- 172.21.15.106:0/2991303226 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fb3f009d170 con 0x7fb400108f90 2024-04-28T12:49:31.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.912+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4000635d0 con 0x7fb400108f90 2024-04-28T12:49:31.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.914+0000 7fb3fd600640 1 -- 172.21.15.106:0/2991303226 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7fb3f009ca70 con 0x7fb400108f90 2024-04-28T12:49:31.911 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:31.911 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:31.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb3e40a3c40 msgr2=0x7fb3e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb3e40a3c40 0x7fb3e40a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb4001a0630 tx=0x7fb3e8005f90 comp rx=0 tx=0).stop 2024-04-28T12:49:31.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb400108f90 msgr2=0x7fb40019f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:31.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb400108f90 0x7fb40019f120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000e9c0 tx=0x7fb3f000ee80 comp rx=0 tx=0).stop 2024-04-28T12:49:31.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 shutdown_connections 2024-04-28T12:49:31.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb3e40a3c40 0x7fb3e40a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb40010ab40 0x7fb4001a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb40010a180 0x7fb40019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 --2- 172.21.15.106:0/2991303226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb400108f90 0x7fb40019f120 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:31.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 >> 172.21.15.106:0/2991303226 conn(0x7fb400077f40 msgr2=0x7fb40010d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:31.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 shutdown_connections 2024-04-28T12:49:31.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:31.917+0000 7fb407000640 1 -- 172.21.15.106:0/2991303226 wait complete. 2024-04-28T12:49:31.933 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-28T12:49:31.933 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:49:32.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.143+0000 7f3596600640 1 -- 172.21.15.106:0/3172171385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590108d90 msgr2=0x7f35901091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.143+0000 7f3596600640 1 --2- 172.21.15.106:0/3172171385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590108d90 0x7f35901091b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3580009900 tx=0x7f358002a410 comp rx=0 tx=0).stop 2024-04-28T12:49:32.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.143+0000 7f3596600640 1 -- 172.21.15.106:0/3172171385 shutdown_connections 2024-04-28T12:49:32.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.143+0000 7f3596600640 1 --2- 172.21.15.106:0/3172171385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359010a940 0x7f35901111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.143+0000 7f3596600640 1 --2- 172.21.15.106:0/3172171385 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3590109f80 0x7f359010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.143+0000 7f3596600640 1 --2- 172.21.15.106:0/3172171385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590108d90 0x7f35901091b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.143+0000 7f3596600640 1 -- 172.21.15.106:0/3172171385 >> 172.21.15.106:0/3172171385 conn(0x7f3590077d30 msgr2=0x7f359007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:32.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.143+0000 7f3596600640 1 -- 172.21.15.106:0/3172171385 shutdown_connections 2024-04-28T12:49:32.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.143+0000 7f3596600640 1 -- 172.21.15.106:0/3172171385 wait complete. 2024-04-28T12:49:32.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.144+0000 7f3596600640 1 Processor -- start 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.144+0000 7f3596600640 1 -- start start 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3596600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3590108d90 0x7f359019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3596600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590109f80 0x7f359019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3596600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359010a940 0x7f35901a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3596600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3590114900 con 0x7f3590109f80 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3596600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3590114780 con 0x7f359010a940 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3596600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3590114a80 con 0x7f3590108d90 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3587e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590109f80 0x7f359019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3587e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590109f80 0x7f359019f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43306/0 (socket says 172.21.15.106:43306) 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3587e00640 1 -- 172.21.15.106:0/3002973509 learned_addr learned my addr 172.21.15.106:0/3002973509 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3595200640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3590108d90 0x7f359019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.145+0000 7f3595c00640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359010a940 0x7f35901a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f3587e00640 1 -- 172.21.15.106:0/3002973509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3590108d90 msgr2=0x7f359019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f3587e00640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3590108d90 0x7f359019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f3587e00640 1 -- 172.21.15.106:0/3002973509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359010a940 msgr2=0x7f35901a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f3587e00640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359010a940 0x7f35901a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f3587e00640 1 -- 172.21.15.106:0/3002973509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35901a6be0 con 0x7f3590109f80 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f3587e00640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590109f80 0x7f359019f500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f357800e9f0 tx=0x7f357800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f358e000640 1 -- 172.21.15.106:0/3002973509 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357800cd80 con 0x7f3590109f80 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f358e000640 1 -- 172.21.15.106:0/3002973509 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35780054d0 con 0x7f3590109f80 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f358e000640 1 -- 172.21.15.106:0/3002973509 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578010690 con 0x7f3590109f80 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35901a6ed0 con 0x7f3590109f80 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.146+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35901a7390 con 0x7f3590109f80 2024-04-28T12:49:32.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.148+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3590110180 con 0x7f3590109f80 2024-04-28T12:49:32.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.150+0000 7f358e000640 1 -- 172.21.15.106:0/3002973509 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3578004e90 con 0x7f3590109f80 2024-04-28T12:49:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.151+0000 7f358e000640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35740a3cc0 0x7f35740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.151+0000 7f358e000640 1 -- 172.21.15.106:0/3002973509 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f35780d0b90 con 0x7f3590109f80 2024-04-28T12:49:32.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.151+0000 7f3595200640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35740a3cc0 0x7f35740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:32.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.152+0000 7f3595200640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35740a3cc0 0x7f35740a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3580005ab0 tx=0x7f3580005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:32.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.156+0000 7f358e000640 1 -- 172.21.15.106:0/3002973509 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f357809d170 con 0x7f3590109f80 2024-04-28T12:49:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.353+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3590062d70 con 0x7f35740a3cc0 2024-04-28T12:49:32.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.388+0000 7f358e000640 1 -- 172.21.15.106:0/3002973509 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340395 (secure 0 0 0) 0x7f3590062d70 con 0x7f35740a3cc0 2024-04-28T12:49:32.390 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:32.392 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-28T12:49:32.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.394+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35740a3cc0 msgr2=0x7f35740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.394+0000 7f3596600640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35740a3cc0 0x7f35740a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3580005ab0 tx=0x7f3580005a20 comp rx=0 tx=0).stop 2024-04-28T12:49:32.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.394+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590109f80 msgr2=0x7f359019f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.394+0000 7f3596600640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590109f80 0x7f359019f500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f357800e9f0 tx=0x7f357800edf0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.394+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 shutdown_connections 2024-04-28T12:49:32.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.395+0000 7f3596600640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f35740a3cc0 0x7f35740a6180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.395+0000 7f3596600640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3590109f80 0x7f359019f500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.395+0000 7f3596600640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f359010a940 0x7f35901a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.395+0000 7f3596600640 1 --2- 172.21.15.106:0/3002973509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3590108d90 0x7f359019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.395+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 >> 172.21.15.106:0/3002973509 conn(0x7f3590077d30 msgr2=0x7f359010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:32.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.395+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 shutdown_connections 2024-04-28T12:49:32.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.395+0000 7f3596600640 1 -- 172.21.15.106:0/3002973509 wait complete. 2024-04-28T12:49:32.410 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-04-28T12:49:32.065277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638439,"num_objects":94,"num_object_clones":0,"num_object_copies":283,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":152,"num_read_kb":168,"num_write":392,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":324,"ondisk_log_size":324,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366960,"kb_used_data":13648,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574864,"statfs":{"total":1151940427776,"available":1151564660736,"internally_reserved":0,"allocated":13975552,"data_stored":7589925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43572,"num_objects":70,"num_object_clones":0,"num_object_copies":211,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":26,"num_read_kb":59,"num_write":163,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005392"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801935+0000","last_change":"2024-04-28T12:48:54.979081+0000","last_active":"2024-04-28T12:49:00.801935+0000","last_peered":"2024-04-28T12:49:00.801935+0000","last_clean":"2024-04-28T12:49:00.801935+0000","last_became_active":"2024-04-28T12:48:54.978668+0000","last_became_peered":"2024-04-28T12:48:54.978668+0000","last_unstale":"2024-04-28T12:49:00.801935+0000","last_undegraded":"2024-04-28T12:49:00.801935+0000","last_fullsized":"2024-04-28T12:49:00.801935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:43.345706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.178550+0000","last_change":"2024-04-28T12:48:54.966277+0000","last_active":"2024-04-28T12:49:00.178550+0000","last_peered":"2024-04-28T12:49:00.178550+0000","last_clean":"2024-04-28T12:49:00.178550+0000","last_became_active":"2024-04-28T12:48:54.965880+0000","last_became_peered":"2024-04-28T12:48:54.965880+0000","last_unstale":"2024-04-28T12:49:00.178550+0000","last_undegraded":"2024-04-28T12:49:00.178550+0000","last_fullsized":"2024-04-28T12:49:00.178550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:42:13.166459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177378+0000","last_change":"2024-04-28T12:48:54.966516+0000","last_active":"2024-04-28T12:49:00.177378+0000","last_peered":"2024-04-28T12:49:00.177378+0000","last_clean":"2024-04-28T12:49:00.177378+0000","last_became_active":"2024-04-28T12:48:54.966112+0000","last_became_peered":"2024-04-28T12:48:54.966112+0000","last_unstale":"2024-04-28T12:49:00.177378+0000","last_undegraded":"2024-04-28T12:49:00.177378+0000","last_fullsized":"2024-04-28T12:49:00.177378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:15:05.538389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802000+0000","last_change":"2024-04-28T12:48:54.962718+0000","last_active":"2024-04-28T12:49:00.802000+0000","last_peered":"2024-04-28T12:49:00.802000+0000","last_clean":"2024-04-28T12:49:00.802000+0000","last_became_active":"2024-04-28T12:48:54.962371+0000","last_became_peered":"2024-04-28T12:48:54.962371+0000","last_unstale":"2024-04-28T12:49:00.802000+0000","last_undegraded":"2024-04-28T12:49:00.802000+0000","last_fullsized":"2024-04-28T12:49:00.802000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:55:47.714734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172799+0000","last_change":"2024-04-28T12:48:54.986898+0000","last_active":"2024-04-28T12:49:00.172799+0000","last_peered":"2024-04-28T12:49:00.172799+0000","last_clean":"2024-04-28T12:49:00.172799+0000","last_became_active":"2024-04-28T12:48:54.986550+0000","last_became_peered":"2024-04-28T12:48:54.986550+0000","last_unstale":"2024-04-28T12:49:00.172799+0000","last_undegraded":"2024-04-28T12:49:00.172799+0000","last_fullsized":"2024-04-28T12:49:00.172799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:54:54.942449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801039+0000","last_change":"2024-04-28T12:48:54.981357+0000","last_active":"2024-04-28T12:49:00.801039+0000","last_peered":"2024-04-28T12:49:00.801039+0000","last_clean":"2024-04-28T12:49:00.801039+0000","last_became_active":"2024-04-28T12:48:54.980945+0000","last_became_peered":"2024-04-28T12:48:54.980945+0000","last_unstale":"2024-04-28T12:49:00.801039+0000","last_undegraded":"2024-04-28T12:49:00.801039+0000","last_fullsized":"2024-04-28T12:49:00.801039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:21:51.476872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172816+0000","last_change":"2024-04-28T12:48:54.968930+0000","last_active":"2024-04-28T12:49:00.172816+0000","last_peered":"2024-04-28T12:49:00.172816+0000","last_clean":"2024-04-28T12:49:00.172816+0000","last_became_active":"2024-04-28T12:48:54.968588+0000","last_became_peered":"2024-04-28T12:48:54.968588+0000","last_unstale":"2024-04-28T12:49:00.172816+0000","last_undegraded":"2024-04-28T12:49:00.172816+0000","last_fullsized":"2024-04-28T12:49:00.172816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:39:59.940136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803771+0000","last_change":"2024-04-28T12:48:54.979500+0000","last_active":"2024-04-28T12:49:00.803771+0000","last_peered":"2024-04-28T12:49:00.803771+0000","last_clean":"2024-04-28T12:49:00.803771+0000","last_became_active":"2024-04-28T12:48:54.978541+0000","last_became_peered":"2024-04-28T12:48:54.978541+0000","last_unstale":"2024-04-28T12:49:00.803771+0000","last_undegraded":"2024-04-28T12:49:00.803771+0000","last_fullsized":"2024-04-28T12:49:00.803771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:06:47.550461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802640+0000","last_change":"2024-04-28T12:48:54.963511+0000","last_active":"2024-04-28T12:49:00.802640+0000","last_peered":"2024-04-28T12:49:00.802640+0000","last_clean":"2024-04-28T12:49:00.802640+0000","last_became_active":"2024-04-28T12:48:54.963178+0000","last_became_peered":"2024-04-28T12:48:54.963178+0000","last_unstale":"2024-04-28T12:49:00.802640+0000","last_undegraded":"2024-04-28T12:49:00.802640+0000","last_fullsized":"2024-04-28T12:49:00.802640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:27:01.709235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802141+0000","last_change":"2024-04-28T12:48:54.961425+0000","last_active":"2024-04-28T12:49:00.802141+0000","last_peered":"2024-04-28T12:49:00.802141+0000","last_clean":"2024-04-28T12:49:00.802141+0000","last_became_active":"2024-04-28T12:48:54.961060+0000","last_became_peered":"2024-04-28T12:48:54.961060+0000","last_unstale":"2024-04-28T12:49:00.802141+0000","last_undegraded":"2024-04-28T12:49:00.802141+0000","last_fullsized":"2024-04-28T12:49:00.802141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:49:14.184108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173407+0000","last_change":"2024-04-28T12:48:54.963483+0000","last_active":"2024-04-28T12:49:00.173407+0000","last_peered":"2024-04-28T12:49:00.173407+0000","last_clean":"2024-04-28T12:49:00.173407+0000","last_became_active":"2024-04-28T12:48:54.963146+0000","last_became_peered":"2024-04-28T12:48:54.963146+0000","last_unstale":"2024-04-28T12:49:00.173407+0000","last_undegraded":"2024-04-28T12:49:00.173407+0000","last_fullsized":"2024-04-28T12:49:00.173407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:05:30.313659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801872+0000","last_change":"2024-04-28T12:48:54.961834+0000","last_active":"2024-04-28T12:49:00.801872+0000","last_peered":"2024-04-28T12:49:00.801872+0000","last_clean":"2024-04-28T12:49:00.801872+0000","last_became_active":"2024-04-28T12:48:54.961521+0000","last_became_peered":"2024-04-28T12:48:54.961521+0000","last_unstale":"2024-04-28T12:49:00.801872+0000","last_undegraded":"2024-04-28T12:49:00.801872+0000","last_fullsized":"2024-04-28T12:49:00.801872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:34:57.168391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803183+0000","last_change":"2024-04-28T12:48:54.979673+0000","last_active":"2024-04-28T12:49:00.803183+0000","last_peered":"2024-04-28T12:49:00.803183+0000","last_clean":"2024-04-28T12:49:00.803183+0000","last_became_active":"2024-04-28T12:48:54.976440+0000","last_became_peered":"2024-04-28T12:48:54.976440+0000","last_unstale":"2024-04-28T12:49:00.803183+0000","last_undegraded":"2024-04-28T12:49:00.803183+0000","last_fullsized":"2024-04-28T12:49:00.803183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:46:30.391135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.176660+0000","last_change":"2024-04-28T12:48:54.963108+0000","last_active":"2024-04-28T12:49:00.176660+0000","last_peered":"2024-04-28T12:49:00.176660+0000","last_clean":"2024-04-28T12:49:00.176660+0000","last_became_active":"2024-04-28T12:48:54.962294+0000","last_became_peered":"2024-04-28T12:48:54.962294+0000","last_unstale":"2024-04-28T12:49:00.176660+0000","last_undegraded":"2024-04-28T12:49:00.176660+0000","last_fullsized":"2024-04-28T12:49:00.176660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:10:03.438962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.176657+0000","last_change":"2024-04-28T12:48:54.967990+0000","last_active":"2024-04-28T12:49:00.176657+0000","last_peered":"2024-04-28T12:49:00.176657+0000","last_clean":"2024-04-28T12:49:00.176657+0000","last_became_active":"2024-04-28T12:48:54.967608+0000","last_became_peered":"2024-04-28T12:48:54.967608+0000","last_unstale":"2024-04-28T12:49:00.176657+0000","last_undegraded":"2024-04-28T12:49:00.176657+0000","last_fullsized":"2024-04-28T12:49:00.176657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:24:17.209965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800886+0000","last_change":"2024-04-28T12:48:54.974436+0000","last_active":"2024-04-28T12:49:00.800886+0000","last_peered":"2024-04-28T12:49:00.800886+0000","last_clean":"2024-04-28T12:49:00.800886+0000","last_became_active":"2024-04-28T12:48:54.973950+0000","last_became_peered":"2024-04-28T12:48:54.973950+0000","last_unstale":"2024-04-28T12:49:00.800886+0000","last_undegraded":"2024-04-28T12:49:00.800886+0000","last_fullsized":"2024-04-28T12:49:00.800886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:51:08.957853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801691+0000","last_change":"2024-04-28T12:48:54.979452+0000","last_active":"2024-04-28T12:49:00.801691+0000","last_peered":"2024-04-28T12:49:00.801691+0000","last_clean":"2024-04-28T12:49:00.801691+0000","last_became_active":"2024-04-28T12:48:54.979033+0000","last_became_peered":"2024-04-28T12:48:54.979033+0000","last_unstale":"2024-04-28T12:49:00.801691+0000","last_undegraded":"2024-04-28T12:49:00.801691+0000","last_fullsized":"2024-04-28T12:49:00.801691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:26:46.644763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806099+0000","last_change":"2024-04-28T12:48:54.970775+0000","last_active":"2024-04-28T12:49:00.806099+0000","last_peered":"2024-04-28T12:49:00.806099+0000","last_clean":"2024-04-28T12:49:00.806099+0000","last_became_active":"2024-04-28T12:48:54.970414+0000","last_became_peered":"2024-04-28T12:48:54.970414+0000","last_unstale":"2024-04-28T12:49:00.806099+0000","last_undegraded":"2024-04-28T12:49:00.806099+0000","last_fullsized":"2024-04-28T12:49:00.806099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:49:59.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173978+0000","last_change":"2024-04-28T12:48:54.983460+0000","last_active":"2024-04-28T12:49:00.173978+0000","last_peered":"2024-04-28T12:49:00.173978+0000","last_clean":"2024-04-28T12:49:00.173978+0000","last_became_active":"2024-04-28T12:48:54.983083+0000","last_became_peered":"2024-04-28T12:48:54.983083+0000","last_unstale":"2024-04-28T12:49:00.173978+0000","last_undegraded":"2024-04-28T12:49:00.173978+0000","last_fullsized":"2024-04-28T12:49:00.173978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:56:14.922227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173515+0000","last_change":"2024-04-28T12:48:59.198781+0000","last_active":"2024-04-28T12:49:00.173515+0000","last_peered":"2024-04-28T12:49:00.173515+0000","last_clean":"2024-04-28T12:49:00.173515+0000","last_became_active":"2024-04-28T12:48:59.198393+0000","last_became_peered":"2024-04-28T12:48:59.198393+0000","last_unstale":"2024-04-28T12:49:00.173515+0000","last_undegraded":"2024-04-28T12:49:00.173515+0000","last_fullsized":"2024-04-28T12:49:00.173515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:37:01.888857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'11","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.007873+0000","last_change":"2024-04-28T12:48:54.131557+0000","last_active":"2024-04-28T12:49:28.007873+0000","last_peered":"2024-04-28T12:49:28.007873+0000","last_clean":"2024-04-28T12:49:28.007873+0000","last_became_active":"2024-04-28T12:48:54.128561+0000","last_became_peered":"2024-04-28T12:48:54.128561+0000","last_unstale":"2024-04-28T12:49:28.007873+0000","last_undegraded":"2024-04-28T12:49:28.007873+0000","last_fullsized":"2024-04-28T12:49:28.007873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:19:41.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.807546+0000","last_change":"2024-04-28T12:48:54.978972+0000","last_active":"2024-04-28T12:49:00.807546+0000","last_peered":"2024-04-28T12:49:00.807546+0000","last_clean":"2024-04-28T12:49:00.807546+0000","last_became_active":"2024-04-28T12:48:54.977025+0000","last_became_peered":"2024-04-28T12:48:54.977025+0000","last_unstale":"2024-04-28T12:49:00.807546+0000","last_undegraded":"2024-04-28T12:49:00.807546+0000","last_fullsized":"2024-04-28T12:49:00.807546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:22:54.411569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801232+0000","last_change":"2024-04-28T12:48:59.193410+0000","last_active":"2024-04-28T12:49:00.801232+0000","last_peered":"2024-04-28T12:49:00.801232+0000","last_clean":"2024-04-28T12:49:00.801232+0000","last_became_active":"2024-04-28T12:48:59.192864+0000","last_became_peered":"2024-04-28T12:48:59.192864+0000","last_unstale":"2024-04-28T12:49:00.801232+0000","last_undegraded":"2024-04-28T12:49:00.801232+0000","last_fullsized":"2024-04-28T12:49:00.801232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:21:18.783650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.807900+0000","last_change":"2024-04-28T12:48:54.132973+0000","last_active":"2024-04-28T12:49:00.807900+0000","last_peered":"2024-04-28T12:49:00.807900+0000","last_clean":"2024-04-28T12:49:00.807900+0000","last_became_active":"2024-04-28T12:48:54.132590+0000","last_became_peered":"2024-04-28T12:48:54.132590+0000","last_unstale":"2024-04-28T12:49:00.807900+0000","last_undegraded":"2024-04-28T12:49:00.807900+0000","last_fullsized":"2024-04-28T12:49:00.807900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:50:52.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177559+0000","last_change":"2024-04-28T12:48:54.975902+0000","last_active":"2024-04-28T12:49:00.177559+0000","last_peered":"2024-04-28T12:49:00.177559+0000","last_clean":"2024-04-28T12:49:00.177559+0000","last_became_active":"2024-04-28T12:48:54.975488+0000","last_became_peered":"2024-04-28T12:48:54.975488+0000","last_unstale":"2024-04-28T12:49:00.177559+0000","last_undegraded":"2024-04-28T12:49:00.177559+0000","last_fullsized":"2024-04-28T12:49:00.177559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:15:10.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801008+0000","last_change":"2024-04-28T12:48:59.187912+0000","last_active":"2024-04-28T12:49:00.801008+0000","last_peered":"2024-04-28T12:49:00.801008+0000","last_clean":"2024-04-28T12:49:00.801008+0000","last_became_active":"2024-04-28T12:48:59.187427+0000","last_became_peered":"2024-04-28T12:48:59.187427+0000","last_unstale":"2024-04-28T12:49:00.801008+0000","last_undegraded":"2024-04-28T12:49:00.801008+0000","last_fullsized":"2024-04-28T12:49:00.801008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:27:53.137136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'11","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.026244+0000","last_change":"2024-04-28T12:48:54.117132+0000","last_active":"2024-04-28T12:49:29.026244+0000","last_peered":"2024-04-28T12:49:29.026244+0000","last_clean":"2024-04-28T12:49:29.026244+0000","last_became_active":"2024-04-28T12:48:54.116805+0000","last_became_peered":"2024-04-28T12:48:54.116805+0000","last_unstale":"2024-04-28T12:49:29.026244+0000","last_undegraded":"2024-04-28T12:49:29.026244+0000","last_fullsized":"2024-04-28T12:49:29.026244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:22:52.015199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802412+0000","last_change":"2024-04-28T12:48:54.972975+0000","last_active":"2024-04-28T12:49:00.802412+0000","last_peered":"2024-04-28T12:49:00.802412+0000","last_clean":"2024-04-28T12:49:00.802412+0000","last_became_active":"2024-04-28T12:48:54.972679+0000","last_became_peered":"2024-04-28T12:48:54.972679+0000","last_unstale":"2024-04-28T12:49:00.802412+0000","last_undegraded":"2024-04-28T12:49:00.802412+0000","last_fullsized":"2024-04-28T12:49:00.802412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:22.032459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800771+0000","last_change":"2024-04-28T12:48:59.187828+0000","last_active":"2024-04-28T12:49:00.800771+0000","last_peered":"2024-04-28T12:49:00.800771+0000","last_clean":"2024-04-28T12:49:00.800771+0000","last_became_active":"2024-04-28T12:48:59.187435+0000","last_became_peered":"2024-04-28T12:48:59.187435+0000","last_unstale":"2024-04-28T12:49:00.800771+0000","last_undegraded":"2024-04-28T12:49:00.800771+0000","last_fullsized":"2024-04-28T12:49:00.800771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:45:22.909378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.025746+0000","last_change":"2024-04-28T12:48:54.112455+0000","last_active":"2024-04-28T12:49:29.025746+0000","last_peered":"2024-04-28T12:49:29.025746+0000","last_clean":"2024-04-28T12:49:29.025746+0000","last_became_active":"2024-04-28T12:48:54.111927+0000","last_became_peered":"2024-04-28T12:48:54.111927+0000","last_unstale":"2024-04-28T12:49:29.025746+0000","last_undegraded":"2024-04-28T12:49:29.025746+0000","last_fullsized":"2024-04-28T12:49:29.025746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:32:54.650426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800856+0000","last_change":"2024-04-28T12:48:54.959687+0000","last_active":"2024-04-28T12:49:00.800856+0000","last_peered":"2024-04-28T12:49:00.800856+0000","last_clean":"2024-04-28T12:49:00.800856+0000","last_became_active":"2024-04-28T12:48:54.959355+0000","last_became_peered":"2024-04-28T12:48:54.959355+0000","last_unstale":"2024-04-28T12:49:00.800856+0000","last_undegraded":"2024-04-28T12:49:00.800856+0000","last_fullsized":"2024-04-28T12:49:00.800856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:53:53.919155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173596+0000","last_change":"2024-04-28T12:48:59.174209+0000","last_active":"2024-04-28T12:49:00.173596+0000","last_peered":"2024-04-28T12:49:00.173596+0000","last_clean":"2024-04-28T12:49:00.173596+0000","last_became_active":"2024-04-28T12:48:59.173700+0000","last_became_peered":"2024-04-28T12:48:59.173700+0000","last_unstale":"2024-04-28T12:49:00.173596+0000","last_undegraded":"2024-04-28T12:49:00.173596+0000","last_fullsized":"2024-04-28T12:49:00.173596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:19:11.820482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.984744+0000","last_change":"2024-04-28T12:48:54.118061+0000","last_active":"2024-04-28T12:49:25.984744+0000","last_peered":"2024-04-28T12:49:25.984744+0000","last_clean":"2024-04-28T12:49:25.984744+0000","last_became_active":"2024-04-28T12:48:54.117663+0000","last_became_peered":"2024-04-28T12:48:54.117663+0000","last_unstale":"2024-04-28T12:49:25.984744+0000","last_undegraded":"2024-04-28T12:49:25.984744+0000","last_fullsized":"2024-04-28T12:49:25.984744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:41:45.206989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173085+0000","last_change":"2024-04-28T12:48:54.975039+0000","last_active":"2024-04-28T12:49:00.173085+0000","last_peered":"2024-04-28T12:49:00.173085+0000","last_clean":"2024-04-28T12:49:00.173085+0000","last_became_active":"2024-04-28T12:48:54.974589+0000","last_became_peered":"2024-04-28T12:48:54.974589+0000","last_unstale":"2024-04-28T12:49:00.173085+0000","last_undegraded":"2024-04-28T12:49:00.173085+0000","last_fullsized":"2024-04-28T12:49:00.173085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:30.550231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173435+0000","last_change":"2024-04-28T12:48:59.183009+0000","last_active":"2024-04-28T12:49:00.173435+0000","last_peered":"2024-04-28T12:49:00.173435+0000","last_clean":"2024-04-28T12:49:00.173435+0000","last_became_active":"2024-04-28T12:48:59.182641+0000","last_became_peered":"2024-04-28T12:48:59.182641+0000","last_unstale":"2024-04-28T12:49:00.173435+0000","last_undegraded":"2024-04-28T12:49:00.173435+0000","last_fullsized":"2024-04-28T12:49:00.173435+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:21.059396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.005861+0000","last_change":"2024-04-28T12:48:54.120770+0000","last_active":"2024-04-28T12:49:28.005861+0000","last_peered":"2024-04-28T12:49:28.005861+0000","last_clean":"2024-04-28T12:49:28.005861+0000","last_became_active":"2024-04-28T12:48:54.120429+0000","last_became_peered":"2024-04-28T12:48:54.120429+0000","last_unstale":"2024-04-28T12:49:28.005861+0000","last_undegraded":"2024-04-28T12:49:28.005861+0000","last_fullsized":"2024-04-28T12:49:28.005861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:27:28.172879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801432+0000","last_change":"2024-04-28T12:48:54.982623+0000","last_active":"2024-04-28T12:49:00.801432+0000","last_peered":"2024-04-28T12:49:00.801432+0000","last_clean":"2024-04-28T12:49:00.801432+0000","last_became_active":"2024-04-28T12:48:54.982225+0000","last_became_peered":"2024-04-28T12:48:54.982225+0000","last_unstale":"2024-04-28T12:49:00.801432+0000","last_undegraded":"2024-04-28T12:49:00.801432+0000","last_fullsized":"2024-04-28T12:49:00.801432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:40:17.652783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801768+0000","last_change":"2024-04-28T12:48:59.188489+0000","last_active":"2024-04-28T12:49:00.801768+0000","last_peered":"2024-04-28T12:49:00.801768+0000","last_clean":"2024-04-28T12:49:00.801768+0000","last_became_active":"2024-04-28T12:48:59.188008+0000","last_became_peered":"2024-04-28T12:48:59.188008+0000","last_unstale":"2024-04-28T12:49:00.801768+0000","last_undegraded":"2024-04-28T12:49:00.801768+0000","last_fullsized":"2024-04-28T12:49:00.801768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:34:55.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:26.999754+0000","last_change":"2024-04-28T12:48:54.137567+0000","last_active":"2024-04-28T12:49:26.999754+0000","last_peered":"2024-04-28T12:49:26.999754+0000","last_clean":"2024-04-28T12:49:26.999754+0000","last_became_active":"2024-04-28T12:48:54.137009+0000","last_became_peered":"2024-04-28T12:48:54.137009+0000","last_unstale":"2024-04-28T12:49:26.999754+0000","last_undegraded":"2024-04-28T12:49:26.999754+0000","last_fullsized":"2024-04-28T12:49:26.999754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:30.909066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803671+0000","last_change":"2024-04-28T12:48:54.974583+0000","last_active":"2024-04-28T12:49:00.803671+0000","last_peered":"2024-04-28T12:49:00.803671+0000","last_clean":"2024-04-28T12:49:00.803671+0000","last_became_active":"2024-04-28T12:48:54.974217+0000","last_became_peered":"2024-04-28T12:48:54.974217+0000","last_unstale":"2024-04-28T12:49:00.803671+0000","last_undegraded":"2024-04-28T12:49:00.803671+0000","last_fullsized":"2024-04-28T12:49:00.803671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:52:05.768593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.178374+0000","last_change":"2024-04-28T12:48:59.185263+0000","last_active":"2024-04-28T12:49:00.178374+0000","last_peered":"2024-04-28T12:49:00.178374+0000","last_clean":"2024-04-28T12:49:00.178374+0000","last_became_active":"2024-04-28T12:48:59.184423+0000","last_became_peered":"2024-04-28T12:48:59.184423+0000","last_unstale":"2024-04-28T12:49:00.178374+0000","last_undegraded":"2024-04-28T12:49:00.178374+0000","last_fullsized":"2024-04-28T12:49:00.178374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:31:01.029985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.003558+0000","last_change":"2024-04-28T12:48:54.119423+0000","last_active":"2024-04-28T12:49:28.003558+0000","last_peered":"2024-04-28T12:49:28.003558+0000","last_clean":"2024-04-28T12:49:28.003558+0000","last_became_active":"2024-04-28T12:48:54.119110+0000","last_became_peered":"2024-04-28T12:48:54.119110+0000","last_unstale":"2024-04-28T12:49:28.003558+0000","last_undegraded":"2024-04-28T12:49:28.003558+0000","last_fullsized":"2024-04-28T12:49:28.003558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:16:33.181397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803826+0000","last_change":"2024-04-28T12:48:54.983978+0000","last_active":"2024-04-28T12:49:00.803826+0000","last_peered":"2024-04-28T12:49:00.803826+0000","last_clean":"2024-04-28T12:49:00.803826+0000","last_became_active":"2024-04-28T12:48:54.982386+0000","last_became_peered":"2024-04-28T12:48:54.982386+0000","last_unstale":"2024-04-28T12:49:00.803826+0000","last_undegraded":"2024-04-28T12:49:00.803826+0000","last_fullsized":"2024-04-28T12:49:00.803826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:23.849403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804506+0000","last_change":"2024-04-28T12:48:59.187771+0000","last_active":"2024-04-28T12:49:00.804506+0000","last_peered":"2024-04-28T12:49:00.804506+0000","last_clean":"2024-04-28T12:49:00.804506+0000","last_became_active":"2024-04-28T12:48:59.187265+0000","last_became_peered":"2024-04-28T12:48:59.187265+0000","last_unstale":"2024-04-28T12:49:00.804506+0000","last_undegraded":"2024-04-28T12:49:00.804506+0000","last_fullsized":"2024-04-28T12:49:00.804506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:51.211998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:31.036343+0000","last_change":"2024-04-28T12:48:54.135666+0000","last_active":"2024-04-28T12:49:31.036343+0000","last_peered":"2024-04-28T12:49:31.036343+0000","last_clean":"2024-04-28T12:49:31.036343+0000","last_became_active":"2024-04-28T12:48:54.135373+0000","last_became_peered":"2024-04-28T12:48:54.135373+0000","last_unstale":"2024-04-28T12:49:31.036343+0000","last_undegraded":"2024-04-28T12:49:31.036343+0000","last_fullsized":"2024-04-28T12:49:31.036343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:41:52.455552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802334+0000","last_change":"2024-04-28T12:48:54.976474+0000","last_active":"2024-04-28T12:49:00.802334+0000","last_peered":"2024-04-28T12:49:00.802334+0000","last_clean":"2024-04-28T12:49:00.802334+0000","last_became_active":"2024-04-28T12:48:54.976133+0000","last_became_peered":"2024-04-28T12:48:54.976133+0000","last_unstale":"2024-04-28T12:49:00.802334+0000","last_undegraded":"2024-04-28T12:49:00.802334+0000","last_fullsized":"2024-04-28T12:49:00.802334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:24:19.046535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803390+0000","last_change":"2024-04-28T12:48:59.181836+0000","last_active":"2024-04-28T12:49:00.803390+0000","last_peered":"2024-04-28T12:49:00.803390+0000","last_clean":"2024-04-28T12:49:00.803390+0000","last_became_active":"2024-04-28T12:48:59.177499+0000","last_became_peered":"2024-04-28T12:48:59.177499+0000","last_unstale":"2024-04-28T12:49:00.803390+0000","last_undegraded":"2024-04-28T12:49:00.803390+0000","last_fullsized":"2024-04-28T12:49:00.803390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:36:37.981250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.024527+0000","last_change":"2024-04-28T12:48:54.118519+0000","last_active":"2024-04-28T12:49:29.024527+0000","last_peered":"2024-04-28T12:49:29.024527+0000","last_clean":"2024-04-28T12:49:29.024527+0000","last_became_active":"2024-04-28T12:48:54.116755+0000","last_became_peered":"2024-04-28T12:48:54.116755+0000","last_unstale":"2024-04-28T12:49:29.024527+0000","last_undegraded":"2024-04-28T12:49:29.024527+0000","last_fullsized":"2024-04-28T12:49:29.024527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:05:46.261098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177763+0000","last_change":"2024-04-28T12:48:54.962931+0000","last_active":"2024-04-28T12:49:00.177763+0000","last_peered":"2024-04-28T12:49:00.177763+0000","last_clean":"2024-04-28T12:49:00.177763+0000","last_became_active":"2024-04-28T12:48:54.961860+0000","last_became_peered":"2024-04-28T12:48:54.961860+0000","last_unstale":"2024-04-28T12:49:00.177763+0000","last_undegraded":"2024-04-28T12:49:00.177763+0000","last_fullsized":"2024-04-28T12:49:00.177763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:28:04.684200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801422+0000","last_change":"2024-04-28T12:48:59.179857+0000","last_active":"2024-04-28T12:49:00.801422+0000","last_peered":"2024-04-28T12:49:00.801422+0000","last_clean":"2024-04-28T12:49:00.801422+0000","last_became_active":"2024-04-28T12:48:59.179412+0000","last_became_peered":"2024-04-28T12:48:59.179412+0000","last_unstale":"2024-04-28T12:49:00.801422+0000","last_undegraded":"2024-04-28T12:49:00.801422+0000","last_fullsized":"2024-04-28T12:49:00.801422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:28:08.883832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.982676+0000","last_change":"2024-04-28T12:48:54.123771+0000","last_active":"2024-04-28T12:49:25.982676+0000","last_peered":"2024-04-28T12:49:25.982676+0000","last_clean":"2024-04-28T12:49:25.982676+0000","last_became_active":"2024-04-28T12:48:54.123524+0000","last_became_peered":"2024-04-28T12:48:54.123524+0000","last_unstale":"2024-04-28T12:49:25.982676+0000","last_undegraded":"2024-04-28T12:49:25.982676+0000","last_fullsized":"2024-04-28T12:49:25.982676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:59:59.982340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802295+0000","last_change":"2024-04-28T12:48:54.966190+0000","last_active":"2024-04-28T12:49:00.802295+0000","last_peered":"2024-04-28T12:49:00.802295+0000","last_clean":"2024-04-28T12:49:00.802295+0000","last_became_active":"2024-04-28T12:48:54.965854+0000","last_became_peered":"2024-04-28T12:48:54.965854+0000","last_unstale":"2024-04-28T12:49:00.802295+0000","last_undegraded":"2024-04-28T12:49:00.802295+0000","last_fullsized":"2024-04-28T12:49:00.802295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:34:16.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805620+0000","last_change":"2024-04-28T12:48:59.182747+0000","last_active":"2024-04-28T12:49:00.805620+0000","last_peered":"2024-04-28T12:49:00.805620+0000","last_clean":"2024-04-28T12:49:00.805620+0000","last_became_active":"2024-04-28T12:48:59.179610+0000","last_became_peered":"2024-04-28T12:48:59.179610+0000","last_unstale":"2024-04-28T12:49:00.805620+0000","last_undegraded":"2024-04-28T12:49:00.805620+0000","last_fullsized":"2024-04-28T12:49:00.805620+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:19:19.603188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'26","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:31.037131+0000","last_change":"2024-04-28T12:48:54.116386+0000","last_active":"2024-04-28T12:49:31.037131+0000","last_peered":"2024-04-28T12:49:31.037131+0000","last_clean":"2024-04-28T12:49:31.037131+0000","last_became_active":"2024-04-28T12:48:54.115547+0000","last_became_peered":"2024-04-28T12:48:54.115547+0000","last_unstale":"2024-04-28T12:49:31.037131+0000","last_undegraded":"2024-04-28T12:49:31.037131+0000","last_fullsized":"2024-04-28T12:49:31.037131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:10:17.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":42,"num_write":28,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173919+0000","last_change":"2024-04-28T12:48:54.962221+0000","last_active":"2024-04-28T12:49:00.173919+0000","last_peered":"2024-04-28T12:49:00.173919+0000","last_clean":"2024-04-28T12:49:00.173919+0000","last_became_active":"2024-04-28T12:48:54.961876+0000","last_became_peered":"2024-04-28T12:48:54.961876+0000","last_unstale":"2024-04-28T12:49:00.173919+0000","last_undegraded":"2024-04-28T12:49:00.173919+0000","last_fullsized":"2024-04-28T12:49:00.173919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:22:38.067688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801307+0000","last_change":"2024-04-28T12:48:59.193852+0000","last_active":"2024-04-28T12:49:00.801307+0000","last_peered":"2024-04-28T12:49:00.801307+0000","last_clean":"2024-04-28T12:49:00.801307+0000","last_became_active":"2024-04-28T12:48:59.193100+0000","last_became_peered":"2024-04-28T12:48:59.193100+0000","last_unstale":"2024-04-28T12:49:00.801307+0000","last_undegraded":"2024-04-28T12:49:00.801307+0000","last_fullsized":"2024-04-28T12:49:00.801307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:35:26.732962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:30.031219+0000","last_change":"2024-04-28T12:48:54.129317+0000","last_active":"2024-04-28T12:49:30.031219+0000","last_peered":"2024-04-28T12:49:30.031219+0000","last_clean":"2024-04-28T12:49:30.031219+0000","last_became_active":"2024-04-28T12:48:54.128980+0000","last_became_peered":"2024-04-28T12:48:54.128980+0000","last_unstale":"2024-04-28T12:49:30.031219+0000","last_undegraded":"2024-04-28T12:49:30.031219+0000","last_fullsized":"2024-04-28T12:49:30.031219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:02:34.543478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802979+0000","last_change":"2024-04-28T12:48:54.983576+0000","last_active":"2024-04-28T12:49:00.802979+0000","last_peered":"2024-04-28T12:49:00.802979+0000","last_clean":"2024-04-28T12:49:00.802979+0000","last_became_active":"2024-04-28T12:48:54.983213+0000","last_became_peered":"2024-04-28T12:48:54.983213+0000","last_unstale":"2024-04-28T12:49:00.802979+0000","last_undegraded":"2024-04-28T12:49:00.802979+0000","last_fullsized":"2024-04-28T12:49:00.802979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:22:10.901393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801118+0000","last_change":"2024-04-28T12:48:59.191085+0000","last_active":"2024-04-28T12:49:00.801118+0000","last_peered":"2024-04-28T12:49:00.801118+0000","last_clean":"2024-04-28T12:49:00.801118+0000","last_became_active":"2024-04-28T12:48:59.190431+0000","last_became_peered":"2024-04-28T12:48:59.190431+0000","last_unstale":"2024-04-28T12:49:00.801118+0000","last_undegraded":"2024-04-28T12:49:00.801118+0000","last_fullsized":"2024-04-28T12:49:00.801118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:13:30.990816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.009959+0000","last_change":"2024-04-28T12:48:54.129634+0000","last_active":"2024-04-28T12:49:28.009959+0000","last_peered":"2024-04-28T12:49:28.009959+0000","last_clean":"2024-04-28T12:49:28.009959+0000","last_became_active":"2024-04-28T12:48:54.129250+0000","last_became_peered":"2024-04-28T12:48:54.129250+0000","last_unstale":"2024-04-28T12:49:28.009959+0000","last_undegraded":"2024-04-28T12:49:28.009959+0000","last_fullsized":"2024-04-28T12:49:28.009959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:48:23.828967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173084+0000","last_change":"2024-04-28T12:48:54.978551+0000","last_active":"2024-04-28T12:49:00.173084+0000","last_peered":"2024-04-28T12:49:00.173084+0000","last_clean":"2024-04-28T12:49:00.173084+0000","last_became_active":"2024-04-28T12:48:54.978193+0000","last_became_peered":"2024-04-28T12:48:54.978193+0000","last_unstale":"2024-04-28T12:49:00.173084+0000","last_undegraded":"2024-04-28T12:49:00.173084+0000","last_fullsized":"2024-04-28T12:49:00.173084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:05:31.289174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806167+0000","last_change":"2024-04-28T12:48:59.195185+0000","last_active":"2024-04-28T12:49:00.806167+0000","last_peered":"2024-04-28T12:49:00.806167+0000","last_clean":"2024-04-28T12:49:00.806167+0000","last_became_active":"2024-04-28T12:48:59.194694+0000","last_became_peered":"2024-04-28T12:48:59.194694+0000","last_unstale":"2024-04-28T12:49:00.806167+0000","last_undegraded":"2024-04-28T12:49:00.806167+0000","last_fullsized":"2024-04-28T12:49:00.806167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:20:32.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.003884+0000","last_change":"2024-04-28T12:48:54.115249+0000","last_active":"2024-04-28T12:49:28.003884+0000","last_peered":"2024-04-28T12:49:28.003884+0000","last_clean":"2024-04-28T12:49:28.003884+0000","last_became_active":"2024-04-28T12:48:54.114888+0000","last_became_peered":"2024-04-28T12:48:54.114888+0000","last_unstale":"2024-04-28T12:49:28.003884+0000","last_undegraded":"2024-04-28T12:49:28.003884+0000","last_fullsized":"2024-04-28T12:49:28.003884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:30.899021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172895+0000","last_change":"2024-04-28T12:48:54.960038+0000","last_active":"2024-04-28T12:49:00.172895+0000","last_peered":"2024-04-28T12:49:00.172895+0000","last_clean":"2024-04-28T12:49:00.172895+0000","last_became_active":"2024-04-28T12:48:54.959710+0000","last_became_peered":"2024-04-28T12:48:54.959710+0000","last_unstale":"2024-04-28T12:49:00.172895+0000","last_undegraded":"2024-04-28T12:49:00.172895+0000","last_fullsized":"2024-04-28T12:49:00.172895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:13.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805376+0000","last_change":"2024-04-28T12:48:59.197523+0000","last_active":"2024-04-28T12:49:00.805376+0000","last_peered":"2024-04-28T12:49:00.805376+0000","last_clean":"2024-04-28T12:49:00.805376+0000","last_became_active":"2024-04-28T12:48:59.197088+0000","last_became_peered":"2024-04-28T12:48:59.197088+0000","last_unstale":"2024-04-28T12:49:00.805376+0000","last_undegraded":"2024-04-28T12:49:00.805376+0000","last_fullsized":"2024-04-28T12:49:00.805376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:53:33.391084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.029935+0000","last_change":"2024-04-28T12:48:54.130359+0000","last_active":"2024-04-28T12:49:29.029935+0000","last_peered":"2024-04-28T12:49:29.029935+0000","last_clean":"2024-04-28T12:49:29.029935+0000","last_became_active":"2024-04-28T12:48:54.130043+0000","last_became_peered":"2024-04-28T12:48:54.130043+0000","last_unstale":"2024-04-28T12:49:29.029935+0000","last_undegraded":"2024-04-28T12:49:29.029935+0000","last_fullsized":"2024-04-28T12:49:29.029935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:05:41.353359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800613+0000","last_change":"2024-04-28T12:48:54.960796+0000","last_active":"2024-04-28T12:49:00.800613+0000","last_peered":"2024-04-28T12:49:00.800613+0000","last_clean":"2024-04-28T12:49:00.800613+0000","last_became_active":"2024-04-28T12:48:54.960457+0000","last_became_peered":"2024-04-28T12:48:54.960457+0000","last_unstale":"2024-04-28T12:49:00.800613+0000","last_undegraded":"2024-04-28T12:49:00.800613+0000","last_fullsized":"2024-04-28T12:49:00.800613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:42:04.096242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800594+0000","last_change":"2024-04-28T12:48:59.193963+0000","last_active":"2024-04-28T12:49:00.800594+0000","last_peered":"2024-04-28T12:49:00.800594+0000","last_clean":"2024-04-28T12:49:00.800594+0000","last_became_active":"2024-04-28T12:48:59.193551+0000","last_became_peered":"2024-04-28T12:48:59.193551+0000","last_unstale":"2024-04-28T12:49:00.800594+0000","last_undegraded":"2024-04-28T12:49:00.800594+0000","last_fullsized":"2024-04-28T12:49:00.800594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:24:29.474296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:27.991325+0000","last_change":"2024-04-28T12:48:54.109277+0000","last_active":"2024-04-28T12:49:27.991325+0000","last_peered":"2024-04-28T12:49:27.991325+0000","last_clean":"2024-04-28T12:49:27.991325+0000","last_became_active":"2024-04-28T12:48:54.108931+0000","last_became_peered":"2024-04-28T12:48:54.108931+0000","last_unstale":"2024-04-28T12:49:27.991325+0000","last_undegraded":"2024-04-28T12:49:27.991325+0000","last_fullsized":"2024-04-28T12:49:27.991325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:56:29.683311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801688+0000","last_change":"2024-04-28T12:48:54.961014+0000","last_active":"2024-04-28T12:49:00.801688+0000","last_peered":"2024-04-28T12:49:00.801688+0000","last_clean":"2024-04-28T12:49:00.801688+0000","last_became_active":"2024-04-28T12:48:54.960648+0000","last_became_peered":"2024-04-28T12:48:54.960648+0000","last_unstale":"2024-04-28T12:49:00.801688+0000","last_undegraded":"2024-04-28T12:49:00.801688+0000","last_fullsized":"2024-04-28T12:49:00.801688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:02:42.794451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172996+0000","last_change":"2024-04-28T12:48:59.185458+0000","last_active":"2024-04-28T12:49:00.172996+0000","last_peered":"2024-04-28T12:49:00.172996+0000","last_clean":"2024-04-28T12:49:00.172996+0000","last_became_active":"2024-04-28T12:48:59.184959+0000","last_became_peered":"2024-04-28T12:48:59.184959+0000","last_unstale":"2024-04-28T12:49:00.172996+0000","last_undegraded":"2024-04-28T12:49:00.172996+0000","last_fullsized":"2024-04-28T12:49:00.172996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:46:45.723374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804442+0000","last_change":"2024-04-28T12:48:54.134303+0000","last_active":"2024-04-28T12:49:00.804442+0000","last_peered":"2024-04-28T12:49:00.804442+0000","last_clean":"2024-04-28T12:49:00.804442+0000","last_became_active":"2024-04-28T12:48:54.133728+0000","last_became_peered":"2024-04-28T12:48:54.133728+0000","last_unstale":"2024-04-28T12:49:00.804442+0000","last_undegraded":"2024-04-28T12:49:00.804442+0000","last_fullsized":"2024-04-28T12:49:00.804442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:44:16.444168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801555+0000","last_change":"2024-04-28T12:48:54.983312+0000","last_active":"2024-04-28T12:49:00.801555+0000","last_peered":"2024-04-28T12:49:00.801555+0000","last_clean":"2024-04-28T12:49:00.801555+0000","last_became_active":"2024-04-28T12:48:54.982928+0000","last_became_peered":"2024-04-28T12:48:54.982928+0000","last_unstale":"2024-04-28T12:49:00.801555+0000","last_undegraded":"2024-04-28T12:49:00.801555+0000","last_fullsized":"2024-04-28T12:49:00.801555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:18:22.411048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805788+0000","last_change":"2024-04-28T12:48:59.187641+0000","last_active":"2024-04-28T12:49:00.805788+0000","last_peered":"2024-04-28T12:49:00.805788+0000","last_clean":"2024-04-28T12:49:00.805788+0000","last_became_active":"2024-04-28T12:48:59.186003+0000","last_became_peered":"2024-04-28T12:48:59.186003+0000","last_unstale":"2024-04-28T12:49:00.805788+0000","last_undegraded":"2024-04-28T12:49:00.805788+0000","last_fullsized":"2024-04-28T12:49:00.805788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:50:34.968976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:30.036912+0000","last_change":"2024-04-28T12:48:54.122909+0000","last_active":"2024-04-28T12:49:30.036912+0000","last_peered":"2024-04-28T12:49:30.036912+0000","last_clean":"2024-04-28T12:49:30.036912+0000","last_became_active":"2024-04-28T12:48:54.122661+0000","last_became_peered":"2024-04-28T12:48:54.122661+0000","last_unstale":"2024-04-28T12:49:30.036912+0000","last_undegraded":"2024-04-28T12:49:30.036912+0000","last_fullsized":"2024-04-28T12:49:30.036912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:43:06.119038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800924+0000","last_change":"2024-04-28T12:48:54.956618+0000","last_active":"2024-04-28T12:49:00.800924+0000","last_peered":"2024-04-28T12:49:00.800924+0000","last_clean":"2024-04-28T12:49:00.800924+0000","last_became_active":"2024-04-28T12:48:54.956227+0000","last_became_peered":"2024-04-28T12:48:54.956227+0000","last_unstale":"2024-04-28T12:49:00.800924+0000","last_undegraded":"2024-04-28T12:49:00.800924+0000","last_fullsized":"2024-04-28T12:49:00.800924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:38:02.820771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806512+0000","last_change":"2024-04-28T12:48:59.187017+0000","last_active":"2024-04-28T12:49:00.806512+0000","last_peered":"2024-04-28T12:49:00.806512+0000","last_clean":"2024-04-28T12:49:00.806512+0000","last_became_active":"2024-04-28T12:48:59.186623+0000","last_became_peered":"2024-04-28T12:48:59.186623+0000","last_unstale":"2024-04-28T12:49:00.806512+0000","last_undegraded":"2024-04-28T12:49:00.806512+0000","last_fullsized":"2024-04-28T12:49:00.806512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:26:53.631400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800721+0000","last_change":"2024-04-28T12:48:54.957348+0000","last_active":"2024-04-28T12:49:00.800721+0000","last_peered":"2024-04-28T12:49:00.800721+0000","last_clean":"2024-04-28T12:49:00.800721+0000","last_became_active":"2024-04-28T12:48:54.956890+0000","last_became_peered":"2024-04-28T12:48:54.956890+0000","last_unstale":"2024-04-28T12:49:00.800721+0000","last_undegraded":"2024-04-28T12:49:00.800721+0000","last_fullsized":"2024-04-28T12:49:00.800721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:41:24.550175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:27.001889+0000","last_change":"2024-04-28T12:48:54.133596+0000","last_active":"2024-04-28T12:49:27.001889+0000","last_peered":"2024-04-28T12:49:27.001889+0000","last_clean":"2024-04-28T12:49:27.001889+0000","last_became_active":"2024-04-28T12:48:54.131632+0000","last_became_peered":"2024-04-28T12:48:54.131632+0000","last_unstale":"2024-04-28T12:49:27.001889+0000","last_undegraded":"2024-04-28T12:49:27.001889+0000","last_fullsized":"2024-04-28T12:49:27.001889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:42:39.352397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173467+0000","last_change":"2024-04-28T12:48:59.174117+0000","last_active":"2024-04-28T12:49:00.173467+0000","last_peered":"2024-04-28T12:49:00.173467+0000","last_clean":"2024-04-28T12:49:00.173467+0000","last_became_active":"2024-04-28T12:48:59.173699+0000","last_became_peered":"2024-04-28T12:48:59.173699+0000","last_unstale":"2024-04-28T12:49:00.173467+0000","last_undegraded":"2024-04-28T12:49:00.173467+0000","last_fullsized":"2024-04-28T12:49:00.173467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:41:51.186413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805626+0000","last_change":"2024-04-28T12:48:54.978658+0000","last_active":"2024-04-28T12:49:00.805626+0000","last_peered":"2024-04-28T12:49:00.805626+0000","last_clean":"2024-04-28T12:49:00.805626+0000","last_became_active":"2024-04-28T12:48:54.978113+0000","last_became_peered":"2024-04-28T12:48:54.978113+0000","last_unstale":"2024-04-28T12:49:00.805626+0000","last_undegraded":"2024-04-28T12:49:00.805626+0000","last_fullsized":"2024-04-28T12:49:00.805626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:33:50.716489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.007267+0000","last_change":"2024-04-28T12:48:54.130301+0000","last_active":"2024-04-28T12:49:28.007267+0000","last_peered":"2024-04-28T12:49:28.007267+0000","last_clean":"2024-04-28T12:49:28.007267+0000","last_became_active":"2024-04-28T12:48:54.130027+0000","last_became_peered":"2024-04-28T12:48:54.130027+0000","last_unstale":"2024-04-28T12:49:28.007267+0000","last_undegraded":"2024-04-28T12:49:28.007267+0000","last_fullsized":"2024-04-28T12:49:28.007267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:30:03.883731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":380,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172744+0000","last_change":"2024-04-28T12:47:34.946674+0000","last_active":"2024-04-28T12:49:00.172744+0000","last_peered":"2024-04-28T12:49:00.172744+0000","last_clean":"2024-04-28T12:49:00.172744+0000","last_became_active":"2024-04-28T12:47:34.946149+0000","last_became_peered":"2024-04-28T12:47:34.946149+0000","last_unstale":"2024-04-28T12:49:00.172744+0000","last_undegraded":"2024-04-28T12:49:00.172744+0000","last_fullsized":"2024-04-28T12:49:00.172744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:43:40.694612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:43:40.694612+0000","last_clean_scrub_stamp":"2024-04-28T12:43:40.694612+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:25:44.776805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800634+0000","last_change":"2024-04-28T12:48:59.190783+0000","last_active":"2024-04-28T12:49:00.800634+0000","last_peered":"2024-04-28T12:49:00.800634+0000","last_clean":"2024-04-28T12:49:00.800634+0000","last_became_active":"2024-04-28T12:48:59.190383+0000","last_became_peered":"2024-04-28T12:48:59.190383+0000","last_unstale":"2024-04-28T12:49:00.800634+0000","last_undegraded":"2024-04-28T12:49:00.800634+0000","last_fullsized":"2024-04-28T12:49:00.800634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:39:41.711227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802259+0000","last_change":"2024-04-28T12:48:54.979911+0000","last_active":"2024-04-28T12:49:00.802259+0000","last_peered":"2024-04-28T12:49:00.802259+0000","last_clean":"2024-04-28T12:49:00.802259+0000","last_became_active":"2024-04-28T12:48:54.974708+0000","last_became_peered":"2024-04-28T12:48:54.974708+0000","last_unstale":"2024-04-28T12:49:00.802259+0000","last_undegraded":"2024-04-28T12:49:00.802259+0000","last_fullsized":"2024-04-28T12:49:00.802259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:48:54.919849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:26.996348+0000","last_change":"2024-04-28T12:48:54.116384+0000","last_active":"2024-04-28T12:49:26.996348+0000","last_peered":"2024-04-28T12:49:26.996348+0000","last_clean":"2024-04-28T12:49:26.996348+0000","last_became_active":"2024-04-28T12:48:54.114362+0000","last_became_peered":"2024-04-28T12:48:54.114362+0000","last_unstale":"2024-04-28T12:49:26.996348+0000","last_undegraded":"2024-04-28T12:49:26.996348+0000","last_fullsized":"2024-04-28T12:49:26.996348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:20.406641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801295+0000","last_change":"2024-04-28T12:48:59.181370+0000","last_active":"2024-04-28T12:49:00.801295+0000","last_peered":"2024-04-28T12:49:00.801295+0000","last_clean":"2024-04-28T12:49:00.801295+0000","last_became_active":"2024-04-28T12:48:59.180852+0000","last_became_peered":"2024-04-28T12:48:59.180852+0000","last_unstale":"2024-04-28T12:49:00.801295+0000","last_undegraded":"2024-04-28T12:49:00.801295+0000","last_fullsized":"2024-04-28T12:49:00.801295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:08:55.532273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.033551+0000","last_change":"2024-04-28T12:48:54.129792+0000","last_active":"2024-04-28T12:49:29.033551+0000","last_peered":"2024-04-28T12:49:29.033551+0000","last_clean":"2024-04-28T12:49:29.033551+0000","last_became_active":"2024-04-28T12:48:54.129437+0000","last_became_peered":"2024-04-28T12:48:54.129437+0000","last_unstale":"2024-04-28T12:49:29.033551+0000","last_undegraded":"2024-04-28T12:49:29.033551+0000","last_fullsized":"2024-04-28T12:49:29.033551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:22:14.658847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804735+0000","last_change":"2024-04-28T12:48:54.980482+0000","last_active":"2024-04-28T12:49:00.804735+0000","last_peered":"2024-04-28T12:49:00.804735+0000","last_clean":"2024-04-28T12:49:00.804735+0000","last_became_active":"2024-04-28T12:48:54.980022+0000","last_became_peered":"2024-04-28T12:48:54.980022+0000","last_unstale":"2024-04-28T12:49:00.804735+0000","last_undegraded":"2024-04-28T12:49:00.804735+0000","last_fullsized":"2024-04-28T12:49:00.804735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:50:11.573598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803612+0000","last_change":"2024-04-28T12:48:59.188749+0000","last_active":"2024-04-28T12:49:00.803612+0000","last_peered":"2024-04-28T12:49:00.803612+0000","last_clean":"2024-04-28T12:49:00.803612+0000","last_became_active":"2024-04-28T12:48:59.187735+0000","last_became_peered":"2024-04-28T12:48:59.187735+0000","last_unstale":"2024-04-28T12:49:00.803612+0000","last_undegraded":"2024-04-28T12:49:00.803612+0000","last_fullsized":"2024-04-28T12:49:00.803612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:50:42.948378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.009741+0000","last_change":"2024-04-28T12:48:54.132690+0000","last_active":"2024-04-28T12:49:28.009741+0000","last_peered":"2024-04-28T12:49:28.009741+0000","last_clean":"2024-04-28T12:49:28.009741+0000","last_became_active":"2024-04-28T12:48:54.132300+0000","last_became_peered":"2024-04-28T12:48:54.132300+0000","last_unstale":"2024-04-28T12:49:28.009741+0000","last_undegraded":"2024-04-28T12:49:28.009741+0000","last_fullsized":"2024-04-28T12:49:28.009741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:14.194355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.807793+0000","last_change":"2024-04-28T12:48:54.981277+0000","last_active":"2024-04-28T12:49:00.807793+0000","last_peered":"2024-04-28T12:49:00.807793+0000","last_clean":"2024-04-28T12:49:00.807793+0000","last_became_active":"2024-04-28T12:48:54.977366+0000","last_became_peered":"2024-04-28T12:48:54.977366+0000","last_unstale":"2024-04-28T12:49:00.807793+0000","last_undegraded":"2024-04-28T12:49:00.807793+0000","last_fullsized":"2024-04-28T12:49:00.807793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:25.323001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172969+0000","last_change":"2024-04-28T12:48:59.174010+0000","last_active":"2024-04-28T12:49:00.172969+0000","last_peered":"2024-04-28T12:49:00.172969+0000","last_clean":"2024-04-28T12:49:00.172969+0000","last_became_active":"2024-04-28T12:48:59.173370+0000","last_became_peered":"2024-04-28T12:48:59.173370+0000","last_unstale":"2024-04-28T12:49:00.172969+0000","last_undegraded":"2024-04-28T12:49:00.172969+0000","last_fullsized":"2024-04-28T12:49:00.172969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:07:43.258318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.031617+0000","last_change":"2024-04-28T12:48:54.127134+0000","last_active":"2024-04-28T12:49:29.031617+0000","last_peered":"2024-04-28T12:49:29.031617+0000","last_clean":"2024-04-28T12:49:29.031617+0000","last_became_active":"2024-04-28T12:48:54.126845+0000","last_became_peered":"2024-04-28T12:48:54.126845+0000","last_unstale":"2024-04-28T12:49:29.031617+0000","last_undegraded":"2024-04-28T12:49:29.031617+0000","last_fullsized":"2024-04-28T12:49:29.031617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:43.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.178342+0000","last_change":"2024-04-28T12:48:54.986671+0000","last_active":"2024-04-28T12:49:00.178342+0000","last_peered":"2024-04-28T12:49:00.178342+0000","last_clean":"2024-04-28T12:49:00.178342+0000","last_became_active":"2024-04-28T12:48:54.985410+0000","last_became_peered":"2024-04-28T12:48:54.985410+0000","last_unstale":"2024-04-28T12:49:00.178342+0000","last_undegraded":"2024-04-28T12:49:00.178342+0000","last_fullsized":"2024-04-28T12:49:00.178342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:15:56.354272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801131+0000","last_change":"2024-04-28T12:48:59.188456+0000","last_active":"2024-04-28T12:49:00.801131+0000","last_peered":"2024-04-28T12:49:00.801131+0000","last_clean":"2024-04-28T12:49:00.801131+0000","last_became_active":"2024-04-28T12:48:59.187984+0000","last_became_peered":"2024-04-28T12:48:59.187984+0000","last_unstale":"2024-04-28T12:49:00.801131+0000","last_undegraded":"2024-04-28T12:49:00.801131+0000","last_fullsized":"2024-04-28T12:49:00.801131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:38:32.653563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.983690+0000","last_change":"2024-04-28T12:48:54.125483+0000","last_active":"2024-04-28T12:49:25.983690+0000","last_peered":"2024-04-28T12:49:25.983690+0000","last_clean":"2024-04-28T12:49:25.983690+0000","last_became_active":"2024-04-28T12:48:54.125261+0000","last_became_peered":"2024-04-28T12:48:54.125261+0000","last_unstale":"2024-04-28T12:49:25.983690+0000","last_undegraded":"2024-04-28T12:49:25.983690+0000","last_fullsized":"2024-04-28T12:49:25.983690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:32:24.986128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172700+0000","last_change":"2024-04-28T12:48:54.960662+0000","last_active":"2024-04-28T12:49:00.172700+0000","last_peered":"2024-04-28T12:49:00.172700+0000","last_clean":"2024-04-28T12:49:00.172700+0000","last_became_active":"2024-04-28T12:48:54.960295+0000","last_became_peered":"2024-04-28T12:48:54.960295+0000","last_unstale":"2024-04-28T12:49:00.172700+0000","last_undegraded":"2024-04-28T12:49:00.172700+0000","last_fullsized":"2024-04-28T12:49:00.172700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:26.190901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806754+0000","last_change":"2024-04-28T12:48:59.194137+0000","last_active":"2024-04-28T12:49:00.806754+0000","last_peered":"2024-04-28T12:49:00.806754+0000","last_clean":"2024-04-28T12:49:00.806754+0000","last_became_active":"2024-04-28T12:48:59.193740+0000","last_became_peered":"2024-04-28T12:48:59.193740+0000","last_unstale":"2024-04-28T12:49:00.806754+0000","last_undegraded":"2024-04-28T12:49:00.806754+0000","last_fullsized":"2024-04-28T12:49:00.806754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:58:52.962196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.984733+0000","last_change":"2024-04-28T12:48:54.140883+0000","last_active":"2024-04-28T12:49:25.984733+0000","last_peered":"2024-04-28T12:49:25.984733+0000","last_clean":"2024-04-28T12:49:25.984733+0000","last_became_active":"2024-04-28T12:48:54.140548+0000","last_became_peered":"2024-04-28T12:48:54.140548+0000","last_unstale":"2024-04-28T12:49:25.984733+0000","last_undegraded":"2024-04-28T12:49:25.984733+0000","last_fullsized":"2024-04-28T12:49:25.984733+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:31.465266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800767+0000","last_change":"2024-04-28T12:48:54.976359+0000","last_active":"2024-04-28T12:49:00.800767+0000","last_peered":"2024-04-28T12:49:00.800767+0000","last_clean":"2024-04-28T12:49:00.800767+0000","last_became_active":"2024-04-28T12:48:54.976028+0000","last_became_peered":"2024-04-28T12:48:54.976028+0000","last_unstale":"2024-04-28T12:49:00.800767+0000","last_undegraded":"2024-04-28T12:49:00.800767+0000","last_fullsized":"2024-04-28T12:49:00.800767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:02:48.498276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802625+0000","last_change":"2024-04-28T12:48:59.179755+0000","last_active":"2024-04-28T12:49:00.802625+0000","last_peered":"2024-04-28T12:49:00.802625+0000","last_clean":"2024-04-28T12:49:00.802625+0000","last_became_active":"2024-04-28T12:48:59.179258+0000","last_became_peered":"2024-04-28T12:48:59.179258+0000","last_unstale":"2024-04-28T12:49:00.802625+0000","last_undegraded":"2024-04-28T12:49:00.802625+0000","last_fullsized":"2024-04-28T12:49:00.802625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:59:38.863875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.009580+0000","last_change":"2024-04-28T12:48:54.121894+0000","last_active":"2024-04-28T12:49:28.009580+0000","last_peered":"2024-04-28T12:49:28.009580+0000","last_clean":"2024-04-28T12:49:28.009580+0000","last_became_active":"2024-04-28T12:48:54.121605+0000","last_became_peered":"2024-04-28T12:48:54.121605+0000","last_unstale":"2024-04-28T12:49:28.009580+0000","last_undegraded":"2024-04-28T12:49:28.009580+0000","last_fullsized":"2024-04-28T12:49:28.009580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:55:11.181943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806914+0000","last_change":"2024-04-28T12:48:54.990809+0000","last_active":"2024-04-28T12:49:00.806914+0000","last_peered":"2024-04-28T12:49:00.806914+0000","last_clean":"2024-04-28T12:49:00.806914+0000","last_became_active":"2024-04-28T12:48:54.990251+0000","last_became_peered":"2024-04-28T12:48:54.990251+0000","last_unstale":"2024-04-28T12:49:00.806914+0000","last_undegraded":"2024-04-28T12:49:00.806914+0000","last_fullsized":"2024-04-28T12:49:00.806914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:14:30.017815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802181+0000","last_change":"2024-04-28T12:48:59.193287+0000","last_active":"2024-04-28T12:49:00.802181+0000","last_peered":"2024-04-28T12:49:00.802181+0000","last_clean":"2024-04-28T12:49:00.802181+0000","last_became_active":"2024-04-28T12:48:59.192755+0000","last_became_peered":"2024-04-28T12:48:59.192755+0000","last_unstale":"2024-04-28T12:49:00.802181+0000","last_undegraded":"2024-04-28T12:49:00.802181+0000","last_fullsized":"2024-04-28T12:49:00.802181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:00:33.790384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.981363+0000","last_change":"2024-04-28T12:48:54.121052+0000","last_active":"2024-04-28T12:49:25.981363+0000","last_peered":"2024-04-28T12:49:25.981363+0000","last_clean":"2024-04-28T12:49:25.981363+0000","last_became_active":"2024-04-28T12:48:54.120702+0000","last_became_peered":"2024-04-28T12:48:54.120702+0000","last_unstale":"2024-04-28T12:49:25.981363+0000","last_undegraded":"2024-04-28T12:49:25.981363+0000","last_fullsized":"2024-04-28T12:49:25.981363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:20:45.771466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172991+0000","last_change":"2024-04-28T12:48:54.970270+0000","last_active":"2024-04-28T12:49:00.172991+0000","last_peered":"2024-04-28T12:49:00.172991+0000","last_clean":"2024-04-28T12:49:00.172991+0000","last_became_active":"2024-04-28T12:48:54.970010+0000","last_became_peered":"2024-04-28T12:48:54.970010+0000","last_unstale":"2024-04-28T12:49:00.172991+0000","last_undegraded":"2024-04-28T12:49:00.172991+0000","last_fullsized":"2024-04-28T12:49:00.172991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:05:03.578751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800870+0000","last_change":"2024-04-28T12:48:59.186197+0000","last_active":"2024-04-28T12:49:00.800870+0000","last_peered":"2024-04-28T12:49:00.800870+0000","last_clean":"2024-04-28T12:49:00.800870+0000","last_became_active":"2024-04-28T12:48:59.185749+0000","last_became_peered":"2024-04-28T12:48:59.185749+0000","last_unstale":"2024-04-28T12:49:00.800870+0000","last_undegraded":"2024-04-28T12:49:00.800870+0000","last_fullsized":"2024-04-28T12:49:00.800870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:36.336775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:30.035394+0000","last_change":"2024-04-28T12:48:54.118725+0000","last_active":"2024-04-28T12:49:30.035394+0000","last_peered":"2024-04-28T12:49:30.035394+0000","last_clean":"2024-04-28T12:49:30.035394+0000","last_became_active":"2024-04-28T12:48:54.118419+0000","last_became_peered":"2024-04-28T12:48:54.118419+0000","last_unstale":"2024-04-28T12:49:30.035394+0000","last_undegraded":"2024-04-28T12:49:30.035394+0000","last_fullsized":"2024-04-28T12:49:30.035394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:25:17.959905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173521+0000","last_change":"2024-04-28T12:48:54.959518+0000","last_active":"2024-04-28T12:49:00.173521+0000","last_peered":"2024-04-28T12:49:00.173521+0000","last_clean":"2024-04-28T12:49:00.173521+0000","last_became_active":"2024-04-28T12:48:54.959101+0000","last_became_peered":"2024-04-28T12:48:54.959101+0000","last_unstale":"2024-04-28T12:49:00.173521+0000","last_undegraded":"2024-04-28T12:49:00.173521+0000","last_fullsized":"2024-04-28T12:49:00.173521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:38:50.496961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177195+0000","last_change":"2024-04-28T12:48:59.180941+0000","last_active":"2024-04-28T12:49:00.177195+0000","last_peered":"2024-04-28T12:49:00.177195+0000","last_clean":"2024-04-28T12:49:00.177195+0000","last_became_active":"2024-04-28T12:48:59.176889+0000","last_became_peered":"2024-04-28T12:48:59.176889+0000","last_unstale":"2024-04-28T12:49:00.177195+0000","last_undegraded":"2024-04-28T12:49:00.177195+0000","last_fullsized":"2024-04-28T12:49:00.177195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:28:21.853493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.009956+0000","last_change":"2024-04-28T12:48:54.126265+0000","last_active":"2024-04-28T12:49:28.009956+0000","last_peered":"2024-04-28T12:49:28.009956+0000","last_clean":"2024-04-28T12:49:28.009956+0000","last_became_active":"2024-04-28T12:48:54.125918+0000","last_became_peered":"2024-04-28T12:48:54.125918+0000","last_unstale":"2024-04-28T12:49:28.009956+0000","last_undegraded":"2024-04-28T12:49:28.009956+0000","last_fullsized":"2024-04-28T12:49:28.009956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:24:26.340143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173220+0000","last_change":"2024-04-28T12:48:54.985335+0000","last_active":"2024-04-28T12:49:00.173220+0000","last_peered":"2024-04-28T12:49:00.173220+0000","last_clean":"2024-04-28T12:49:00.173220+0000","last_became_active":"2024-04-28T12:48:54.985017+0000","last_became_peered":"2024-04-28T12:48:54.985017+0000","last_unstale":"2024-04-28T12:49:00.173220+0000","last_undegraded":"2024-04-28T12:49:00.173220+0000","last_fullsized":"2024-04-28T12:49:00.173220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:49:21.821976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800227+0000","last_change":"2024-04-28T12:48:59.176438+0000","last_active":"2024-04-28T12:49:00.800227+0000","last_peered":"2024-04-28T12:49:00.800227+0000","last_clean":"2024-04-28T12:49:00.800227+0000","last_became_active":"2024-04-28T12:48:59.175773+0000","last_became_peered":"2024-04-28T12:48:59.175773+0000","last_unstale":"2024-04-28T12:49:00.800227+0000","last_undegraded":"2024-04-28T12:49:00.800227+0000","last_fullsized":"2024-04-28T12:49:00.800227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:32.687553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805149+0000","last_change":"2024-04-28T12:48:54.130452+0000","last_active":"2024-04-28T12:49:00.805149+0000","last_peered":"2024-04-28T12:49:00.805149+0000","last_clean":"2024-04-28T12:49:00.805149+0000","last_became_active":"2024-04-28T12:48:54.130090+0000","last_became_peered":"2024-04-28T12:48:54.130090+0000","last_unstale":"2024-04-28T12:49:00.805149+0000","last_undegraded":"2024-04-28T12:49:00.805149+0000","last_fullsized":"2024-04-28T12:49:00.805149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:02:34.014710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173492+0000","last_change":"2024-04-28T12:48:54.961838+0000","last_active":"2024-04-28T12:49:00.173492+0000","last_peered":"2024-04-28T12:49:00.173492+0000","last_clean":"2024-04-28T12:49:00.173492+0000","last_became_active":"2024-04-28T12:48:54.961525+0000","last_became_peered":"2024-04-28T12:48:54.961525+0000","last_unstale":"2024-04-28T12:49:00.173492+0000","last_undegraded":"2024-04-28T12:49:00.173492+0000","last_fullsized":"2024-04-28T12:49:00.173492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:06:46.077023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800883+0000","last_change":"2024-04-28T12:48:54.959806+0000","last_active":"2024-04-28T12:49:00.800883+0000","last_peered":"2024-04-28T12:49:00.800883+0000","last_clean":"2024-04-28T12:49:00.800883+0000","last_became_active":"2024-04-28T12:48:54.959491+0000","last_became_peered":"2024-04-28T12:48:54.959491+0000","last_unstale":"2024-04-28T12:49:00.800883+0000","last_undegraded":"2024-04-28T12:49:00.800883+0000","last_fullsized":"2024-04-28T12:49:00.800883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:12:25.367185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.174176+0000","last_change":"2024-04-28T12:48:54.960035+0000","last_active":"2024-04-28T12:49:00.174176+0000","last_peered":"2024-04-28T12:49:00.174176+0000","last_clean":"2024-04-28T12:49:00.174176+0000","last_became_active":"2024-04-28T12:48:54.959684+0000","last_became_peered":"2024-04-28T12:48:54.959684+0000","last_unstale":"2024-04-28T12:49:00.174176+0000","last_undegraded":"2024-04-28T12:49:00.174176+0000","last_fullsized":"2024-04-28T12:49:00.174176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:46:36.484928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173089+0000","last_change":"2024-04-28T12:48:54.984682+0000","last_active":"2024-04-28T12:49:00.173089+0000","last_peered":"2024-04-28T12:49:00.173089+0000","last_clean":"2024-04-28T12:49:00.173089+0000","last_became_active":"2024-04-28T12:48:54.984346+0000","last_became_peered":"2024-04-28T12:48:54.984346+0000","last_unstale":"2024-04-28T12:49:00.173089+0000","last_undegraded":"2024-04-28T12:49:00.173089+0000","last_fullsized":"2024-04-28T12:49:00.173089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:22:57.828641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177475+0000","last_change":"2024-04-28T12:48:54.981526+0000","last_active":"2024-04-28T12:49:00.177475+0000","last_peered":"2024-04-28T12:49:00.177475+0000","last_clean":"2024-04-28T12:49:00.177475+0000","last_became_active":"2024-04-28T12:48:54.981212+0000","last_became_peered":"2024-04-28T12:48:54.981212+0000","last_unstale":"2024-04-28T12:49:00.177475+0000","last_undegraded":"2024-04-28T12:49:00.177475+0000","last_fullsized":"2024-04-28T12:49:00.177475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:29:00.976903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804836+0000","last_change":"2024-04-28T12:48:54.986664+0000","last_active":"2024-04-28T12:49:00.804836+0000","last_peered":"2024-04-28T12:49:00.804836+0000","last_clean":"2024-04-28T12:49:00.804836+0000","last_became_active":"2024-04-28T12:48:54.986358+0000","last_became_peered":"2024-04-28T12:48:54.986358+0000","last_unstale":"2024-04-28T12:49:00.804836+0000","last_undegraded":"2024-04-28T12:49:00.804836+0000","last_fullsized":"2024-04-28T12:49:00.804836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:11:08.970414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801788+0000","last_change":"2024-04-28T12:48:54.964342+0000","last_active":"2024-04-28T12:49:00.801788+0000","last_peered":"2024-04-28T12:49:00.801788+0000","last_clean":"2024-04-28T12:49:00.801788+0000","last_became_active":"2024-04-28T12:48:54.963980+0000","last_became_peered":"2024-04-28T12:48:54.963980+0000","last_unstale":"2024-04-28T12:49:00.801788+0000","last_undegraded":"2024-04-28T12:49:00.801788+0000","last_fullsized":"2024-04-28T12:49:00.801788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:17:07.772838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173837+0000","last_change":"2024-04-28T12:48:54.963955+0000","last_active":"2024-04-28T12:49:00.173837+0000","last_peered":"2024-04-28T12:49:00.173837+0000","last_clean":"2024-04-28T12:49:00.173837+0000","last_became_active":"2024-04-28T12:48:54.963572+0000","last_became_peered":"2024-04-28T12:48:54.963572+0000","last_unstale":"2024-04-28T12:49:00.173837+0000","last_undegraded":"2024-04-28T12:49:00.173837+0000","last_fullsized":"2024-04-28T12:49:00.173837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:18.077455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.807988+0000","last_change":"2024-04-28T12:48:54.978143+0000","last_active":"2024-04-28T12:49:00.807988+0000","last_peered":"2024-04-28T12:49:00.807988+0000","last_clean":"2024-04-28T12:49:00.807988+0000","last_became_active":"2024-04-28T12:48:54.977715+0000","last_became_peered":"2024-04-28T12:48:54.977715+0000","last_unstale":"2024-04-28T12:49:00.807988+0000","last_undegraded":"2024-04-28T12:49:00.807988+0000","last_fullsized":"2024-04-28T12:49:00.807988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:47:17.071081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800439+0000","last_change":"2024-04-28T12:48:54.982613+0000","last_active":"2024-04-28T12:49:00.800439+0000","last_peered":"2024-04-28T12:49:00.800439+0000","last_clean":"2024-04-28T12:49:00.800439+0000","last_became_active":"2024-04-28T12:48:54.982231+0000","last_became_peered":"2024-04-28T12:48:54.982231+0000","last_unstale":"2024-04-28T12:49:00.800439+0000","last_undegraded":"2024-04-28T12:49:00.800439+0000","last_fullsized":"2024-04-28T12:49:00.800439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:22.206377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804186+0000","last_change":"2024-04-28T12:48:54.978398+0000","last_active":"2024-04-28T12:49:00.804186+0000","last_peered":"2024-04-28T12:49:00.804186+0000","last_clean":"2024-04-28T12:49:00.804186+0000","last_became_active":"2024-04-28T12:48:54.977015+0000","last_became_peered":"2024-04-28T12:48:54.977015+0000","last_unstale":"2024-04-28T12:49:00.804186+0000","last_undegraded":"2024-04-28T12:49:00.804186+0000","last_fullsized":"2024-04-28T12:49:00.804186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:39:39.499850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801910+0000","last_change":"2024-04-28T12:48:54.982870+0000","last_active":"2024-04-28T12:49:00.801910+0000","last_peered":"2024-04-28T12:49:00.801910+0000","last_clean":"2024-04-28T12:49:00.801910+0000","last_became_active":"2024-04-28T12:48:54.982308+0000","last_became_peered":"2024-04-28T12:48:54.982308+0000","last_unstale":"2024-04-28T12:49:00.801910+0000","last_undegraded":"2024-04-28T12:49:00.801910+0000","last_fullsized":"2024-04-28T12:49:00.801910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:47:45.976825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802980+0000","last_change":"2024-04-28T12:48:54.987904+0000","last_active":"2024-04-28T12:49:00.802980+0000","last_peered":"2024-04-28T12:49:00.802980+0000","last_clean":"2024-04-28T12:49:00.802980+0000","last_became_active":"2024-04-28T12:48:54.987551+0000","last_became_peered":"2024-04-28T12:48:54.987551+0000","last_unstale":"2024-04-28T12:49:00.802980+0000","last_undegraded":"2024-04-28T12:49:00.802980+0000","last_fullsized":"2024-04-28T12:49:00.802980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:30:36.831005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801542+0000","last_change":"2024-04-28T12:48:54.963176+0000","last_active":"2024-04-28T12:49:00.801542+0000","last_peered":"2024-04-28T12:49:00.801542+0000","last_clean":"2024-04-28T12:49:00.801542+0000","last_became_active":"2024-04-28T12:48:54.962830+0000","last_became_peered":"2024-04-28T12:48:54.962830+0000","last_unstale":"2024-04-28T12:49:00.801542+0000","last_undegraded":"2024-04-28T12:49:00.801542+0000","last_fullsized":"2024-04-28T12:49:00.801542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:10:40.671376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47954,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":26,"num_read_kb":59,"num_write":210,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":142815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383700,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":477756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645337,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1092810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808862,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":477992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":478674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332140,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1613824,"data_stored":1075562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":477264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":502067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":479743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":503069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":477368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":477840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36916,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708236,"statfs":{"total":95995035648,"available":95957233664,"internally_reserved":0,"allocated":1617920,"data_stored":1069780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":42598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:49:32.417 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-28T12:49:32.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:32 smithi171 ceph-mon[63127]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:32.501 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:32 smithi171 ceph-mon[63127]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:32.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:32 smithi171 ceph-mon[63127]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-28T12:49:32.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:32 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/22764538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:49:32.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:32 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2991303226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:32.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:32 smithi129 ceph-mon[62418]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:32.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:32 smithi129 ceph-mon[62418]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:32.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:32 smithi129 ceph-mon[62418]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-28T12:49:32.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:32 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/22764538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:49:32.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:32 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2991303226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:32.524 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:32 smithi106 ceph-mon[28081]: from='client.24842 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:32.524 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:32 smithi106 ceph-mon[28081]: from='client.24848 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:49:32.524 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:32 smithi106 ceph-mon[28081]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-28T12:49:32.524 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:32 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/22764538' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:49:32.525 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:32 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2991303226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:32.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.663+0000 7ffae1e00640 1 -- 172.21.15.106:0/1631887319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffadc108450 msgr2=0x7ffadc10a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.663+0000 7ffae1e00640 1 --2- 172.21.15.106:0/1631887319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffadc108450 0x7ffadc10a8a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffacc009900 tx=0x7ffacc02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.663+0000 7ffae1e00640 1 -- 172.21.15.106:0/1631887319 shutdown_connections 2024-04-28T12:49:32.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.663+0000 7ffae1e00640 1 --2- 172.21.15.106:0/1631887319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffadc108450 0x7ffadc10a8a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.663+0000 7ffae1e00640 1 --2- 172.21.15.106:0/1631887319 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffadc105b20 0x7ffadc107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.659 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.663+0000 7ffae1e00640 1 --2- 172.21.15.106:0/1631887319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffadc1031f0 0x7ffadc1055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.663+0000 7ffae1e00640 1 -- 172.21.15.106:0/1631887319 >> 172.21.15.106:0/1631887319 conn(0x7ffadc0fcbc0 msgr2=0x7ffadc0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:32.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.663+0000 7ffae1e00640 1 -- 172.21.15.106:0/1631887319 shutdown_connections 2024-04-28T12:49:32.660 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.663+0000 7ffae1e00640 1 -- 172.21.15.106:0/1631887319 wait complete. 2024-04-28T12:49:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.664+0000 7ffae1e00640 1 Processor -- start 2024-04-28T12:49:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.664+0000 7ffae1e00640 1 -- start start 2024-04-28T12:49:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.664+0000 7ffae1e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffadc1031f0 0x7ffadc19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffae1e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffadc105b20 0x7ffadc19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:32.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffae1e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffadc108450 0x7ffadc1a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffae1e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc10e2c0 con 0x7ffadc108450 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffae1e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc10e140 con 0x7ffadc1031f0 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffae1e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc10e440 con 0x7ffadc105b20 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffadc105b20 0x7ffadc19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffadc105b20 0x7ffadc19b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:50454/0 (socket says 172.21.15.106:50454) 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffae1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffadc108450 0x7ffadc1a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffae0a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffadc1031f0 0x7ffadc19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffadbe00640 1 -- 172.21.15.106:0/3693828153 learned_addr learned my addr 172.21.15.106:0/3693828153 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffadbe00640 1 -- 172.21.15.106:0/3693828153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffadc1031f0 msgr2=0x7ffadc19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffadbe00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffadc1031f0 0x7ffadc19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffadbe00640 1 -- 172.21.15.106:0/3693828153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffadc108450 msgr2=0x7ffadc1a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffadbe00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffadc108450 0x7ffadc1a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.665+0000 7ffadbe00640 1 -- 172.21.15.106:0/3693828153 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc1a2730 con 0x7ffadc105b20 2024-04-28T12:49:32.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.666+0000 7ffadbe00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffadc105b20 0x7ffadc19b050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffac400e9f0 tx=0x7ffac400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:32.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.666+0000 7ffad9600640 1 -- 172.21.15.106:0/3693828153 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac400ccc0 con 0x7ffadc105b20 2024-04-28T12:49:32.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.666+0000 7ffad9600640 1 -- 172.21.15.106:0/3693828153 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac40054d0 con 0x7ffadc105b20 2024-04-28T12:49:32.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.666+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc1a2a20 con 0x7ffadc105b20 2024-04-28T12:49:32.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.666+0000 7ffad9600640 1 -- 172.21.15.106:0/3693828153 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac4010430 con 0x7ffadc105b20 2024-04-28T12:49:32.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.666+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc1a2f20 con 0x7ffadc105b20 2024-04-28T12:49:32.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.667+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffadc109f40 con 0x7ffadc105b20 2024-04-28T12:49:32.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.669+0000 7ffad9600640 1 -- 172.21.15.106:0/3693828153 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ffac40046c0 con 0x7ffadc105b20 2024-04-28T12:49:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.670+0000 7ffad9600640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffac00a3c40 0x7ffac00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:32.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.671+0000 7ffad9600640 1 -- 172.21.15.106:0/3693828153 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ffac40d0ca0 con 0x7ffadc105b20 2024-04-28T12:49:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.671+0000 7ffae0a00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffac00a3c40 0x7ffac00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:32.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.671+0000 7ffae0a00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffac00a3c40 0x7ffac00a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffad4005fd0 tx=0x7ffad4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:32.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.673+0000 7ffad9600640 1 -- 172.21.15.106:0/3693828153 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ffac409d280 con 0x7ffadc105b20 2024-04-28T12:49:32.870 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.874+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffadc062d70 con 0x7ffac00a3c40 2024-04-28T12:49:32.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.896+0000 7ffad9600640 1 -- 172.21.15.106:0/3693828153 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340395 (secure 0 0 0) 0x7ffadc062d70 con 0x7ffac00a3c40 2024-04-28T12:49:32.898 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:32.900 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffac00a3c40 msgr2=0x7ffac00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffac00a3c40 0x7ffac00a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffad4005fd0 tx=0x7ffad4009500 comp rx=0 tx=0).stop 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffadc105b20 msgr2=0x7ffadc19b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffadc105b20 0x7ffadc19b050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffac400e9f0 tx=0x7ffac400eeb0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 shutdown_connections 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffac00a3c40 0x7ffac00a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffadc108450 0x7ffadc1a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffadc105b20 0x7ffadc19b050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 --2- 172.21.15.106:0/3693828153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffadc1031f0 0x7ffadc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:32.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 >> 172.21.15.106:0/3693828153 conn(0x7ffadc0fcbc0 msgr2=0x7ffadc102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:32.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 shutdown_connections 2024-04-28T12:49:32.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:32.904+0000 7ffae1e00640 1 -- 172.21.15.106:0/3693828153 wait complete. 2024-04-28T12:49:32.922 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-04-28T12:49:32.065277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638439,"num_objects":94,"num_object_clones":0,"num_object_copies":283,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":152,"num_read_kb":168,"num_write":392,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":324,"ondisk_log_size":324,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366960,"kb_used_data":13648,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574864,"statfs":{"total":1151940427776,"available":1151564660736,"internally_reserved":0,"allocated":13975552,"data_stored":7589925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43572,"num_objects":70,"num_object_clones":0,"num_object_copies":211,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":26,"num_read_kb":59,"num_write":163,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":47,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005392"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801935+0000","last_change":"2024-04-28T12:48:54.979081+0000","last_active":"2024-04-28T12:49:00.801935+0000","last_peered":"2024-04-28T12:49:00.801935+0000","last_clean":"2024-04-28T12:49:00.801935+0000","last_became_active":"2024-04-28T12:48:54.978668+0000","last_became_peered":"2024-04-28T12:48:54.978668+0000","last_unstale":"2024-04-28T12:49:00.801935+0000","last_undegraded":"2024-04-28T12:49:00.801935+0000","last_fullsized":"2024-04-28T12:49:00.801935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:43.345706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.178550+0000","last_change":"2024-04-28T12:48:54.966277+0000","last_active":"2024-04-28T12:49:00.178550+0000","last_peered":"2024-04-28T12:49:00.178550+0000","last_clean":"2024-04-28T12:49:00.178550+0000","last_became_active":"2024-04-28T12:48:54.965880+0000","last_became_peered":"2024-04-28T12:48:54.965880+0000","last_unstale":"2024-04-28T12:49:00.178550+0000","last_undegraded":"2024-04-28T12:49:00.178550+0000","last_fullsized":"2024-04-28T12:49:00.178550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:42:13.166459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177378+0000","last_change":"2024-04-28T12:48:54.966516+0000","last_active":"2024-04-28T12:49:00.177378+0000","last_peered":"2024-04-28T12:49:00.177378+0000","last_clean":"2024-04-28T12:49:00.177378+0000","last_became_active":"2024-04-28T12:48:54.966112+0000","last_became_peered":"2024-04-28T12:48:54.966112+0000","last_unstale":"2024-04-28T12:49:00.177378+0000","last_undegraded":"2024-04-28T12:49:00.177378+0000","last_fullsized":"2024-04-28T12:49:00.177378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:15:05.538389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802000+0000","last_change":"2024-04-28T12:48:54.962718+0000","last_active":"2024-04-28T12:49:00.802000+0000","last_peered":"2024-04-28T12:49:00.802000+0000","last_clean":"2024-04-28T12:49:00.802000+0000","last_became_active":"2024-04-28T12:48:54.962371+0000","last_became_peered":"2024-04-28T12:48:54.962371+0000","last_unstale":"2024-04-28T12:49:00.802000+0000","last_undegraded":"2024-04-28T12:49:00.802000+0000","last_fullsized":"2024-04-28T12:49:00.802000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:55:47.714734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172799+0000","last_change":"2024-04-28T12:48:54.986898+0000","last_active":"2024-04-28T12:49:00.172799+0000","last_peered":"2024-04-28T12:49:00.172799+0000","last_clean":"2024-04-28T12:49:00.172799+0000","last_became_active":"2024-04-28T12:48:54.986550+0000","last_became_peered":"2024-04-28T12:48:54.986550+0000","last_unstale":"2024-04-28T12:49:00.172799+0000","last_undegraded":"2024-04-28T12:49:00.172799+0000","last_fullsized":"2024-04-28T12:49:00.172799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:54:54.942449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801039+0000","last_change":"2024-04-28T12:48:54.981357+0000","last_active":"2024-04-28T12:49:00.801039+0000","last_peered":"2024-04-28T12:49:00.801039+0000","last_clean":"2024-04-28T12:49:00.801039+0000","last_became_active":"2024-04-28T12:48:54.980945+0000","last_became_peered":"2024-04-28T12:48:54.980945+0000","last_unstale":"2024-04-28T12:49:00.801039+0000","last_undegraded":"2024-04-28T12:49:00.801039+0000","last_fullsized":"2024-04-28T12:49:00.801039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:21:51.476872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172816+0000","last_change":"2024-04-28T12:48:54.968930+0000","last_active":"2024-04-28T12:49:00.172816+0000","last_peered":"2024-04-28T12:49:00.172816+0000","last_clean":"2024-04-28T12:49:00.172816+0000","last_became_active":"2024-04-28T12:48:54.968588+0000","last_became_peered":"2024-04-28T12:48:54.968588+0000","last_unstale":"2024-04-28T12:49:00.172816+0000","last_undegraded":"2024-04-28T12:49:00.172816+0000","last_fullsized":"2024-04-28T12:49:00.172816+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:39:59.940136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803771+0000","last_change":"2024-04-28T12:48:54.979500+0000","last_active":"2024-04-28T12:49:00.803771+0000","last_peered":"2024-04-28T12:49:00.803771+0000","last_clean":"2024-04-28T12:49:00.803771+0000","last_became_active":"2024-04-28T12:48:54.978541+0000","last_became_peered":"2024-04-28T12:48:54.978541+0000","last_unstale":"2024-04-28T12:49:00.803771+0000","last_undegraded":"2024-04-28T12:49:00.803771+0000","last_fullsized":"2024-04-28T12:49:00.803771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:06:47.550461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802640+0000","last_change":"2024-04-28T12:48:54.963511+0000","last_active":"2024-04-28T12:49:00.802640+0000","last_peered":"2024-04-28T12:49:00.802640+0000","last_clean":"2024-04-28T12:49:00.802640+0000","last_became_active":"2024-04-28T12:48:54.963178+0000","last_became_peered":"2024-04-28T12:48:54.963178+0000","last_unstale":"2024-04-28T12:49:00.802640+0000","last_undegraded":"2024-04-28T12:49:00.802640+0000","last_fullsized":"2024-04-28T12:49:00.802640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:27:01.709235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802141+0000","last_change":"2024-04-28T12:48:54.961425+0000","last_active":"2024-04-28T12:49:00.802141+0000","last_peered":"2024-04-28T12:49:00.802141+0000","last_clean":"2024-04-28T12:49:00.802141+0000","last_became_active":"2024-04-28T12:48:54.961060+0000","last_became_peered":"2024-04-28T12:48:54.961060+0000","last_unstale":"2024-04-28T12:49:00.802141+0000","last_undegraded":"2024-04-28T12:49:00.802141+0000","last_fullsized":"2024-04-28T12:49:00.802141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:49:14.184108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173407+0000","last_change":"2024-04-28T12:48:54.963483+0000","last_active":"2024-04-28T12:49:00.173407+0000","last_peered":"2024-04-28T12:49:00.173407+0000","last_clean":"2024-04-28T12:49:00.173407+0000","last_became_active":"2024-04-28T12:48:54.963146+0000","last_became_peered":"2024-04-28T12:48:54.963146+0000","last_unstale":"2024-04-28T12:49:00.173407+0000","last_undegraded":"2024-04-28T12:49:00.173407+0000","last_fullsized":"2024-04-28T12:49:00.173407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:05:30.313659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801872+0000","last_change":"2024-04-28T12:48:54.961834+0000","last_active":"2024-04-28T12:49:00.801872+0000","last_peered":"2024-04-28T12:49:00.801872+0000","last_clean":"2024-04-28T12:49:00.801872+0000","last_became_active":"2024-04-28T12:48:54.961521+0000","last_became_peered":"2024-04-28T12:48:54.961521+0000","last_unstale":"2024-04-28T12:49:00.801872+0000","last_undegraded":"2024-04-28T12:49:00.801872+0000","last_fullsized":"2024-04-28T12:49:00.801872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:34:57.168391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803183+0000","last_change":"2024-04-28T12:48:54.979673+0000","last_active":"2024-04-28T12:49:00.803183+0000","last_peered":"2024-04-28T12:49:00.803183+0000","last_clean":"2024-04-28T12:49:00.803183+0000","last_became_active":"2024-04-28T12:48:54.976440+0000","last_became_peered":"2024-04-28T12:48:54.976440+0000","last_unstale":"2024-04-28T12:49:00.803183+0000","last_undegraded":"2024-04-28T12:49:00.803183+0000","last_fullsized":"2024-04-28T12:49:00.803183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:46:30.391135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.176660+0000","last_change":"2024-04-28T12:48:54.963108+0000","last_active":"2024-04-28T12:49:00.176660+0000","last_peered":"2024-04-28T12:49:00.176660+0000","last_clean":"2024-04-28T12:49:00.176660+0000","last_became_active":"2024-04-28T12:48:54.962294+0000","last_became_peered":"2024-04-28T12:48:54.962294+0000","last_unstale":"2024-04-28T12:49:00.176660+0000","last_undegraded":"2024-04-28T12:49:00.176660+0000","last_fullsized":"2024-04-28T12:49:00.176660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:10:03.438962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.176657+0000","last_change":"2024-04-28T12:48:54.967990+0000","last_active":"2024-04-28T12:49:00.176657+0000","last_peered":"2024-04-28T12:49:00.176657+0000","last_clean":"2024-04-28T12:49:00.176657+0000","last_became_active":"2024-04-28T12:48:54.967608+0000","last_became_peered":"2024-04-28T12:48:54.967608+0000","last_unstale":"2024-04-28T12:49:00.176657+0000","last_undegraded":"2024-04-28T12:49:00.176657+0000","last_fullsized":"2024-04-28T12:49:00.176657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:24:17.209965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800886+0000","last_change":"2024-04-28T12:48:54.974436+0000","last_active":"2024-04-28T12:49:00.800886+0000","last_peered":"2024-04-28T12:49:00.800886+0000","last_clean":"2024-04-28T12:49:00.800886+0000","last_became_active":"2024-04-28T12:48:54.973950+0000","last_became_peered":"2024-04-28T12:48:54.973950+0000","last_unstale":"2024-04-28T12:49:00.800886+0000","last_undegraded":"2024-04-28T12:49:00.800886+0000","last_fullsized":"2024-04-28T12:49:00.800886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:51:08.957853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801691+0000","last_change":"2024-04-28T12:48:54.979452+0000","last_active":"2024-04-28T12:49:00.801691+0000","last_peered":"2024-04-28T12:49:00.801691+0000","last_clean":"2024-04-28T12:49:00.801691+0000","last_became_active":"2024-04-28T12:48:54.979033+0000","last_became_peered":"2024-04-28T12:48:54.979033+0000","last_unstale":"2024-04-28T12:49:00.801691+0000","last_undegraded":"2024-04-28T12:49:00.801691+0000","last_fullsized":"2024-04-28T12:49:00.801691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:26:46.644763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806099+0000","last_change":"2024-04-28T12:48:54.970775+0000","last_active":"2024-04-28T12:49:00.806099+0000","last_peered":"2024-04-28T12:49:00.806099+0000","last_clean":"2024-04-28T12:49:00.806099+0000","last_became_active":"2024-04-28T12:48:54.970414+0000","last_became_peered":"2024-04-28T12:48:54.970414+0000","last_unstale":"2024-04-28T12:49:00.806099+0000","last_undegraded":"2024-04-28T12:49:00.806099+0000","last_fullsized":"2024-04-28T12:49:00.806099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:49:59.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173978+0000","last_change":"2024-04-28T12:48:54.983460+0000","last_active":"2024-04-28T12:49:00.173978+0000","last_peered":"2024-04-28T12:49:00.173978+0000","last_clean":"2024-04-28T12:49:00.173978+0000","last_became_active":"2024-04-28T12:48:54.983083+0000","last_became_peered":"2024-04-28T12:48:54.983083+0000","last_unstale":"2024-04-28T12:49:00.173978+0000","last_undegraded":"2024-04-28T12:49:00.173978+0000","last_fullsized":"2024-04-28T12:49:00.173978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:56:14.922227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173515+0000","last_change":"2024-04-28T12:48:59.198781+0000","last_active":"2024-04-28T12:49:00.173515+0000","last_peered":"2024-04-28T12:49:00.173515+0000","last_clean":"2024-04-28T12:49:00.173515+0000","last_became_active":"2024-04-28T12:48:59.198393+0000","last_became_peered":"2024-04-28T12:48:59.198393+0000","last_unstale":"2024-04-28T12:49:00.173515+0000","last_undegraded":"2024-04-28T12:49:00.173515+0000","last_fullsized":"2024-04-28T12:49:00.173515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:37:01.888857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'11","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.007873+0000","last_change":"2024-04-28T12:48:54.131557+0000","last_active":"2024-04-28T12:49:28.007873+0000","last_peered":"2024-04-28T12:49:28.007873+0000","last_clean":"2024-04-28T12:49:28.007873+0000","last_became_active":"2024-04-28T12:48:54.128561+0000","last_became_peered":"2024-04-28T12:48:54.128561+0000","last_unstale":"2024-04-28T12:49:28.007873+0000","last_undegraded":"2024-04-28T12:49:28.007873+0000","last_fullsized":"2024-04-28T12:49:28.007873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:19:41.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.807546+0000","last_change":"2024-04-28T12:48:54.978972+0000","last_active":"2024-04-28T12:49:00.807546+0000","last_peered":"2024-04-28T12:49:00.807546+0000","last_clean":"2024-04-28T12:49:00.807546+0000","last_became_active":"2024-04-28T12:48:54.977025+0000","last_became_peered":"2024-04-28T12:48:54.977025+0000","last_unstale":"2024-04-28T12:49:00.807546+0000","last_undegraded":"2024-04-28T12:49:00.807546+0000","last_fullsized":"2024-04-28T12:49:00.807546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:22:54.411569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801232+0000","last_change":"2024-04-28T12:48:59.193410+0000","last_active":"2024-04-28T12:49:00.801232+0000","last_peered":"2024-04-28T12:49:00.801232+0000","last_clean":"2024-04-28T12:49:00.801232+0000","last_became_active":"2024-04-28T12:48:59.192864+0000","last_became_peered":"2024-04-28T12:48:59.192864+0000","last_unstale":"2024-04-28T12:49:00.801232+0000","last_undegraded":"2024-04-28T12:49:00.801232+0000","last_fullsized":"2024-04-28T12:49:00.801232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:21:18.783650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.807900+0000","last_change":"2024-04-28T12:48:54.132973+0000","last_active":"2024-04-28T12:49:00.807900+0000","last_peered":"2024-04-28T12:49:00.807900+0000","last_clean":"2024-04-28T12:49:00.807900+0000","last_became_active":"2024-04-28T12:48:54.132590+0000","last_became_peered":"2024-04-28T12:48:54.132590+0000","last_unstale":"2024-04-28T12:49:00.807900+0000","last_undegraded":"2024-04-28T12:49:00.807900+0000","last_fullsized":"2024-04-28T12:49:00.807900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:50:52.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177559+0000","last_change":"2024-04-28T12:48:54.975902+0000","last_active":"2024-04-28T12:49:00.177559+0000","last_peered":"2024-04-28T12:49:00.177559+0000","last_clean":"2024-04-28T12:49:00.177559+0000","last_became_active":"2024-04-28T12:48:54.975488+0000","last_became_peered":"2024-04-28T12:48:54.975488+0000","last_unstale":"2024-04-28T12:49:00.177559+0000","last_undegraded":"2024-04-28T12:49:00.177559+0000","last_fullsized":"2024-04-28T12:49:00.177559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:15:10.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801008+0000","last_change":"2024-04-28T12:48:59.187912+0000","last_active":"2024-04-28T12:49:00.801008+0000","last_peered":"2024-04-28T12:49:00.801008+0000","last_clean":"2024-04-28T12:49:00.801008+0000","last_became_active":"2024-04-28T12:48:59.187427+0000","last_became_peered":"2024-04-28T12:48:59.187427+0000","last_unstale":"2024-04-28T12:49:00.801008+0000","last_undegraded":"2024-04-28T12:49:00.801008+0000","last_fullsized":"2024-04-28T12:49:00.801008+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:27:53.137136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'11","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.026244+0000","last_change":"2024-04-28T12:48:54.117132+0000","last_active":"2024-04-28T12:49:29.026244+0000","last_peered":"2024-04-28T12:49:29.026244+0000","last_clean":"2024-04-28T12:49:29.026244+0000","last_became_active":"2024-04-28T12:48:54.116805+0000","last_became_peered":"2024-04-28T12:48:54.116805+0000","last_unstale":"2024-04-28T12:49:29.026244+0000","last_undegraded":"2024-04-28T12:49:29.026244+0000","last_fullsized":"2024-04-28T12:49:29.026244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:22:52.015199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802412+0000","last_change":"2024-04-28T12:48:54.972975+0000","last_active":"2024-04-28T12:49:00.802412+0000","last_peered":"2024-04-28T12:49:00.802412+0000","last_clean":"2024-04-28T12:49:00.802412+0000","last_became_active":"2024-04-28T12:48:54.972679+0000","last_became_peered":"2024-04-28T12:48:54.972679+0000","last_unstale":"2024-04-28T12:49:00.802412+0000","last_undegraded":"2024-04-28T12:49:00.802412+0000","last_fullsized":"2024-04-28T12:49:00.802412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:22.032459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800771+0000","last_change":"2024-04-28T12:48:59.187828+0000","last_active":"2024-04-28T12:49:00.800771+0000","last_peered":"2024-04-28T12:49:00.800771+0000","last_clean":"2024-04-28T12:49:00.800771+0000","last_became_active":"2024-04-28T12:48:59.187435+0000","last_became_peered":"2024-04-28T12:48:59.187435+0000","last_unstale":"2024-04-28T12:49:00.800771+0000","last_undegraded":"2024-04-28T12:49:00.800771+0000","last_fullsized":"2024-04-28T12:49:00.800771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:45:22.909378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.025746+0000","last_change":"2024-04-28T12:48:54.112455+0000","last_active":"2024-04-28T12:49:29.025746+0000","last_peered":"2024-04-28T12:49:29.025746+0000","last_clean":"2024-04-28T12:49:29.025746+0000","last_became_active":"2024-04-28T12:48:54.111927+0000","last_became_peered":"2024-04-28T12:48:54.111927+0000","last_unstale":"2024-04-28T12:49:29.025746+0000","last_undegraded":"2024-04-28T12:49:29.025746+0000","last_fullsized":"2024-04-28T12:49:29.025746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:32:54.650426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800856+0000","last_change":"2024-04-28T12:48:54.959687+0000","last_active":"2024-04-28T12:49:00.800856+0000","last_peered":"2024-04-28T12:49:00.800856+0000","last_clean":"2024-04-28T12:49:00.800856+0000","last_became_active":"2024-04-28T12:48:54.959355+0000","last_became_peered":"2024-04-28T12:48:54.959355+0000","last_unstale":"2024-04-28T12:49:00.800856+0000","last_undegraded":"2024-04-28T12:49:00.800856+0000","last_fullsized":"2024-04-28T12:49:00.800856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:53:53.919155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173596+0000","last_change":"2024-04-28T12:48:59.174209+0000","last_active":"2024-04-28T12:49:00.173596+0000","last_peered":"2024-04-28T12:49:00.173596+0000","last_clean":"2024-04-28T12:49:00.173596+0000","last_became_active":"2024-04-28T12:48:59.173700+0000","last_became_peered":"2024-04-28T12:48:59.173700+0000","last_unstale":"2024-04-28T12:49:00.173596+0000","last_undegraded":"2024-04-28T12:49:00.173596+0000","last_fullsized":"2024-04-28T12:49:00.173596+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:19:11.820482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.984744+0000","last_change":"2024-04-28T12:48:54.118061+0000","last_active":"2024-04-28T12:49:25.984744+0000","last_peered":"2024-04-28T12:49:25.984744+0000","last_clean":"2024-04-28T12:49:25.984744+0000","last_became_active":"2024-04-28T12:48:54.117663+0000","last_became_peered":"2024-04-28T12:48:54.117663+0000","last_unstale":"2024-04-28T12:49:25.984744+0000","last_undegraded":"2024-04-28T12:49:25.984744+0000","last_fullsized":"2024-04-28T12:49:25.984744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:41:45.206989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173085+0000","last_change":"2024-04-28T12:48:54.975039+0000","last_active":"2024-04-28T12:49:00.173085+0000","last_peered":"2024-04-28T12:49:00.173085+0000","last_clean":"2024-04-28T12:49:00.173085+0000","last_became_active":"2024-04-28T12:48:54.974589+0000","last_became_peered":"2024-04-28T12:48:54.974589+0000","last_unstale":"2024-04-28T12:49:00.173085+0000","last_undegraded":"2024-04-28T12:49:00.173085+0000","last_fullsized":"2024-04-28T12:49:00.173085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:30.550231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173435+0000","last_change":"2024-04-28T12:48:59.183009+0000","last_active":"2024-04-28T12:49:00.173435+0000","last_peered":"2024-04-28T12:49:00.173435+0000","last_clean":"2024-04-28T12:49:00.173435+0000","last_became_active":"2024-04-28T12:48:59.182641+0000","last_became_peered":"2024-04-28T12:48:59.182641+0000","last_unstale":"2024-04-28T12:49:00.173435+0000","last_undegraded":"2024-04-28T12:49:00.173435+0000","last_fullsized":"2024-04-28T12:49:00.173435+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:21.059396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.005861+0000","last_change":"2024-04-28T12:48:54.120770+0000","last_active":"2024-04-28T12:49:28.005861+0000","last_peered":"2024-04-28T12:49:28.005861+0000","last_clean":"2024-04-28T12:49:28.005861+0000","last_became_active":"2024-04-28T12:48:54.120429+0000","last_became_peered":"2024-04-28T12:48:54.120429+0000","last_unstale":"2024-04-28T12:49:28.005861+0000","last_undegraded":"2024-04-28T12:49:28.005861+0000","last_fullsized":"2024-04-28T12:49:28.005861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:27:28.172879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801432+0000","last_change":"2024-04-28T12:48:54.982623+0000","last_active":"2024-04-28T12:49:00.801432+0000","last_peered":"2024-04-28T12:49:00.801432+0000","last_clean":"2024-04-28T12:49:00.801432+0000","last_became_active":"2024-04-28T12:48:54.982225+0000","last_became_peered":"2024-04-28T12:48:54.982225+0000","last_unstale":"2024-04-28T12:49:00.801432+0000","last_undegraded":"2024-04-28T12:49:00.801432+0000","last_fullsized":"2024-04-28T12:49:00.801432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:40:17.652783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801768+0000","last_change":"2024-04-28T12:48:59.188489+0000","last_active":"2024-04-28T12:49:00.801768+0000","last_peered":"2024-04-28T12:49:00.801768+0000","last_clean":"2024-04-28T12:49:00.801768+0000","last_became_active":"2024-04-28T12:48:59.188008+0000","last_became_peered":"2024-04-28T12:48:59.188008+0000","last_unstale":"2024-04-28T12:49:00.801768+0000","last_undegraded":"2024-04-28T12:49:00.801768+0000","last_fullsized":"2024-04-28T12:49:00.801768+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:34:55.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:26.999754+0000","last_change":"2024-04-28T12:48:54.137567+0000","last_active":"2024-04-28T12:49:26.999754+0000","last_peered":"2024-04-28T12:49:26.999754+0000","last_clean":"2024-04-28T12:49:26.999754+0000","last_became_active":"2024-04-28T12:48:54.137009+0000","last_became_peered":"2024-04-28T12:48:54.137009+0000","last_unstale":"2024-04-28T12:49:26.999754+0000","last_undegraded":"2024-04-28T12:49:26.999754+0000","last_fullsized":"2024-04-28T12:49:26.999754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:30.909066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803671+0000","last_change":"2024-04-28T12:48:54.974583+0000","last_active":"2024-04-28T12:49:00.803671+0000","last_peered":"2024-04-28T12:49:00.803671+0000","last_clean":"2024-04-28T12:49:00.803671+0000","last_became_active":"2024-04-28T12:48:54.974217+0000","last_became_peered":"2024-04-28T12:48:54.974217+0000","last_unstale":"2024-04-28T12:49:00.803671+0000","last_undegraded":"2024-04-28T12:49:00.803671+0000","last_fullsized":"2024-04-28T12:49:00.803671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:52:05.768593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.178374+0000","last_change":"2024-04-28T12:48:59.185263+0000","last_active":"2024-04-28T12:49:00.178374+0000","last_peered":"2024-04-28T12:49:00.178374+0000","last_clean":"2024-04-28T12:49:00.178374+0000","last_became_active":"2024-04-28T12:48:59.184423+0000","last_became_peered":"2024-04-28T12:48:59.184423+0000","last_unstale":"2024-04-28T12:49:00.178374+0000","last_undegraded":"2024-04-28T12:49:00.178374+0000","last_fullsized":"2024-04-28T12:49:00.178374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:31:01.029985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.003558+0000","last_change":"2024-04-28T12:48:54.119423+0000","last_active":"2024-04-28T12:49:28.003558+0000","last_peered":"2024-04-28T12:49:28.003558+0000","last_clean":"2024-04-28T12:49:28.003558+0000","last_became_active":"2024-04-28T12:48:54.119110+0000","last_became_peered":"2024-04-28T12:48:54.119110+0000","last_unstale":"2024-04-28T12:49:28.003558+0000","last_undegraded":"2024-04-28T12:49:28.003558+0000","last_fullsized":"2024-04-28T12:49:28.003558+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:16:33.181397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803826+0000","last_change":"2024-04-28T12:48:54.983978+0000","last_active":"2024-04-28T12:49:00.803826+0000","last_peered":"2024-04-28T12:49:00.803826+0000","last_clean":"2024-04-28T12:49:00.803826+0000","last_became_active":"2024-04-28T12:48:54.982386+0000","last_became_peered":"2024-04-28T12:48:54.982386+0000","last_unstale":"2024-04-28T12:49:00.803826+0000","last_undegraded":"2024-04-28T12:49:00.803826+0000","last_fullsized":"2024-04-28T12:49:00.803826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:23.849403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804506+0000","last_change":"2024-04-28T12:48:59.187771+0000","last_active":"2024-04-28T12:49:00.804506+0000","last_peered":"2024-04-28T12:49:00.804506+0000","last_clean":"2024-04-28T12:49:00.804506+0000","last_became_active":"2024-04-28T12:48:59.187265+0000","last_became_peered":"2024-04-28T12:48:59.187265+0000","last_unstale":"2024-04-28T12:49:00.804506+0000","last_undegraded":"2024-04-28T12:49:00.804506+0000","last_fullsized":"2024-04-28T12:49:00.804506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:51.211998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:31.036343+0000","last_change":"2024-04-28T12:48:54.135666+0000","last_active":"2024-04-28T12:49:31.036343+0000","last_peered":"2024-04-28T12:49:31.036343+0000","last_clean":"2024-04-28T12:49:31.036343+0000","last_became_active":"2024-04-28T12:48:54.135373+0000","last_became_peered":"2024-04-28T12:48:54.135373+0000","last_unstale":"2024-04-28T12:49:31.036343+0000","last_undegraded":"2024-04-28T12:49:31.036343+0000","last_fullsized":"2024-04-28T12:49:31.036343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:41:52.455552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802334+0000","last_change":"2024-04-28T12:48:54.976474+0000","last_active":"2024-04-28T12:49:00.802334+0000","last_peered":"2024-04-28T12:49:00.802334+0000","last_clean":"2024-04-28T12:49:00.802334+0000","last_became_active":"2024-04-28T12:48:54.976133+0000","last_became_peered":"2024-04-28T12:48:54.976133+0000","last_unstale":"2024-04-28T12:49:00.802334+0000","last_undegraded":"2024-04-28T12:49:00.802334+0000","last_fullsized":"2024-04-28T12:49:00.802334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:24:19.046535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803390+0000","last_change":"2024-04-28T12:48:59.181836+0000","last_active":"2024-04-28T12:49:00.803390+0000","last_peered":"2024-04-28T12:49:00.803390+0000","last_clean":"2024-04-28T12:49:00.803390+0000","last_became_active":"2024-04-28T12:48:59.177499+0000","last_became_peered":"2024-04-28T12:48:59.177499+0000","last_unstale":"2024-04-28T12:49:00.803390+0000","last_undegraded":"2024-04-28T12:49:00.803390+0000","last_fullsized":"2024-04-28T12:49:00.803390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:36:37.981250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.024527+0000","last_change":"2024-04-28T12:48:54.118519+0000","last_active":"2024-04-28T12:49:29.024527+0000","last_peered":"2024-04-28T12:49:29.024527+0000","last_clean":"2024-04-28T12:49:29.024527+0000","last_became_active":"2024-04-28T12:48:54.116755+0000","last_became_peered":"2024-04-28T12:48:54.116755+0000","last_unstale":"2024-04-28T12:49:29.024527+0000","last_undegraded":"2024-04-28T12:49:29.024527+0000","last_fullsized":"2024-04-28T12:49:29.024527+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:05:46.261098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177763+0000","last_change":"2024-04-28T12:48:54.962931+0000","last_active":"2024-04-28T12:49:00.177763+0000","last_peered":"2024-04-28T12:49:00.177763+0000","last_clean":"2024-04-28T12:49:00.177763+0000","last_became_active":"2024-04-28T12:48:54.961860+0000","last_became_peered":"2024-04-28T12:48:54.961860+0000","last_unstale":"2024-04-28T12:49:00.177763+0000","last_undegraded":"2024-04-28T12:49:00.177763+0000","last_fullsized":"2024-04-28T12:49:00.177763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:28:04.684200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801422+0000","last_change":"2024-04-28T12:48:59.179857+0000","last_active":"2024-04-28T12:49:00.801422+0000","last_peered":"2024-04-28T12:49:00.801422+0000","last_clean":"2024-04-28T12:49:00.801422+0000","last_became_active":"2024-04-28T12:48:59.179412+0000","last_became_peered":"2024-04-28T12:48:59.179412+0000","last_unstale":"2024-04-28T12:49:00.801422+0000","last_undegraded":"2024-04-28T12:49:00.801422+0000","last_fullsized":"2024-04-28T12:49:00.801422+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:28:08.883832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.982676+0000","last_change":"2024-04-28T12:48:54.123771+0000","last_active":"2024-04-28T12:49:25.982676+0000","last_peered":"2024-04-28T12:49:25.982676+0000","last_clean":"2024-04-28T12:49:25.982676+0000","last_became_active":"2024-04-28T12:48:54.123524+0000","last_became_peered":"2024-04-28T12:48:54.123524+0000","last_unstale":"2024-04-28T12:49:25.982676+0000","last_undegraded":"2024-04-28T12:49:25.982676+0000","last_fullsized":"2024-04-28T12:49:25.982676+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:59:59.982340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802295+0000","last_change":"2024-04-28T12:48:54.966190+0000","last_active":"2024-04-28T12:49:00.802295+0000","last_peered":"2024-04-28T12:49:00.802295+0000","last_clean":"2024-04-28T12:49:00.802295+0000","last_became_active":"2024-04-28T12:48:54.965854+0000","last_became_peered":"2024-04-28T12:48:54.965854+0000","last_unstale":"2024-04-28T12:49:00.802295+0000","last_undegraded":"2024-04-28T12:49:00.802295+0000","last_fullsized":"2024-04-28T12:49:00.802295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:34:16.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805620+0000","last_change":"2024-04-28T12:48:59.182747+0000","last_active":"2024-04-28T12:49:00.805620+0000","last_peered":"2024-04-28T12:49:00.805620+0000","last_clean":"2024-04-28T12:49:00.805620+0000","last_became_active":"2024-04-28T12:48:59.179610+0000","last_became_peered":"2024-04-28T12:48:59.179610+0000","last_unstale":"2024-04-28T12:49:00.805620+0000","last_undegraded":"2024-04-28T12:49:00.805620+0000","last_fullsized":"2024-04-28T12:49:00.805620+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:19:19.603188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'26","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:31.037131+0000","last_change":"2024-04-28T12:48:54.116386+0000","last_active":"2024-04-28T12:49:31.037131+0000","last_peered":"2024-04-28T12:49:31.037131+0000","last_clean":"2024-04-28T12:49:31.037131+0000","last_became_active":"2024-04-28T12:48:54.115547+0000","last_became_peered":"2024-04-28T12:48:54.115547+0000","last_unstale":"2024-04-28T12:49:31.037131+0000","last_undegraded":"2024-04-28T12:49:31.037131+0000","last_fullsized":"2024-04-28T12:49:31.037131+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:10:17.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":7,"num_read_kb":42,"num_write":28,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173919+0000","last_change":"2024-04-28T12:48:54.962221+0000","last_active":"2024-04-28T12:49:00.173919+0000","last_peered":"2024-04-28T12:49:00.173919+0000","last_clean":"2024-04-28T12:49:00.173919+0000","last_became_active":"2024-04-28T12:48:54.961876+0000","last_became_peered":"2024-04-28T12:48:54.961876+0000","last_unstale":"2024-04-28T12:49:00.173919+0000","last_undegraded":"2024-04-28T12:49:00.173919+0000","last_fullsized":"2024-04-28T12:49:00.173919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:22:38.067688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801307+0000","last_change":"2024-04-28T12:48:59.193852+0000","last_active":"2024-04-28T12:49:00.801307+0000","last_peered":"2024-04-28T12:49:00.801307+0000","last_clean":"2024-04-28T12:49:00.801307+0000","last_became_active":"2024-04-28T12:48:59.193100+0000","last_became_peered":"2024-04-28T12:48:59.193100+0000","last_unstale":"2024-04-28T12:49:00.801307+0000","last_undegraded":"2024-04-28T12:49:00.801307+0000","last_fullsized":"2024-04-28T12:49:00.801307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:35:26.732962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:30.031219+0000","last_change":"2024-04-28T12:48:54.129317+0000","last_active":"2024-04-28T12:49:30.031219+0000","last_peered":"2024-04-28T12:49:30.031219+0000","last_clean":"2024-04-28T12:49:30.031219+0000","last_became_active":"2024-04-28T12:48:54.128980+0000","last_became_peered":"2024-04-28T12:48:54.128980+0000","last_unstale":"2024-04-28T12:49:30.031219+0000","last_undegraded":"2024-04-28T12:49:30.031219+0000","last_fullsized":"2024-04-28T12:49:30.031219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:02:34.543478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802979+0000","last_change":"2024-04-28T12:48:54.983576+0000","last_active":"2024-04-28T12:49:00.802979+0000","last_peered":"2024-04-28T12:49:00.802979+0000","last_clean":"2024-04-28T12:49:00.802979+0000","last_became_active":"2024-04-28T12:48:54.983213+0000","last_became_peered":"2024-04-28T12:48:54.983213+0000","last_unstale":"2024-04-28T12:49:00.802979+0000","last_undegraded":"2024-04-28T12:49:00.802979+0000","last_fullsized":"2024-04-28T12:49:00.802979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:22:10.901393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801118+0000","last_change":"2024-04-28T12:48:59.191085+0000","last_active":"2024-04-28T12:49:00.801118+0000","last_peered":"2024-04-28T12:49:00.801118+0000","last_clean":"2024-04-28T12:49:00.801118+0000","last_became_active":"2024-04-28T12:48:59.190431+0000","last_became_peered":"2024-04-28T12:48:59.190431+0000","last_unstale":"2024-04-28T12:49:00.801118+0000","last_undegraded":"2024-04-28T12:49:00.801118+0000","last_fullsized":"2024-04-28T12:49:00.801118+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:13:30.990816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.009959+0000","last_change":"2024-04-28T12:48:54.129634+0000","last_active":"2024-04-28T12:49:28.009959+0000","last_peered":"2024-04-28T12:49:28.009959+0000","last_clean":"2024-04-28T12:49:28.009959+0000","last_became_active":"2024-04-28T12:48:54.129250+0000","last_became_peered":"2024-04-28T12:48:54.129250+0000","last_unstale":"2024-04-28T12:49:28.009959+0000","last_undegraded":"2024-04-28T12:49:28.009959+0000","last_fullsized":"2024-04-28T12:49:28.009959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:48:23.828967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173084+0000","last_change":"2024-04-28T12:48:54.978551+0000","last_active":"2024-04-28T12:49:00.173084+0000","last_peered":"2024-04-28T12:49:00.173084+0000","last_clean":"2024-04-28T12:49:00.173084+0000","last_became_active":"2024-04-28T12:48:54.978193+0000","last_became_peered":"2024-04-28T12:48:54.978193+0000","last_unstale":"2024-04-28T12:49:00.173084+0000","last_undegraded":"2024-04-28T12:49:00.173084+0000","last_fullsized":"2024-04-28T12:49:00.173084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:05:31.289174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806167+0000","last_change":"2024-04-28T12:48:59.195185+0000","last_active":"2024-04-28T12:49:00.806167+0000","last_peered":"2024-04-28T12:49:00.806167+0000","last_clean":"2024-04-28T12:49:00.806167+0000","last_became_active":"2024-04-28T12:48:59.194694+0000","last_became_peered":"2024-04-28T12:48:59.194694+0000","last_unstale":"2024-04-28T12:49:00.806167+0000","last_undegraded":"2024-04-28T12:49:00.806167+0000","last_fullsized":"2024-04-28T12:49:00.806167+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:20:32.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.003884+0000","last_change":"2024-04-28T12:48:54.115249+0000","last_active":"2024-04-28T12:49:28.003884+0000","last_peered":"2024-04-28T12:49:28.003884+0000","last_clean":"2024-04-28T12:49:28.003884+0000","last_became_active":"2024-04-28T12:48:54.114888+0000","last_became_peered":"2024-04-28T12:48:54.114888+0000","last_unstale":"2024-04-28T12:49:28.003884+0000","last_undegraded":"2024-04-28T12:49:28.003884+0000","last_fullsized":"2024-04-28T12:49:28.003884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:30.899021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172895+0000","last_change":"2024-04-28T12:48:54.960038+0000","last_active":"2024-04-28T12:49:00.172895+0000","last_peered":"2024-04-28T12:49:00.172895+0000","last_clean":"2024-04-28T12:49:00.172895+0000","last_became_active":"2024-04-28T12:48:54.959710+0000","last_became_peered":"2024-04-28T12:48:54.959710+0000","last_unstale":"2024-04-28T12:49:00.172895+0000","last_undegraded":"2024-04-28T12:49:00.172895+0000","last_fullsized":"2024-04-28T12:49:00.172895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:13.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805376+0000","last_change":"2024-04-28T12:48:59.197523+0000","last_active":"2024-04-28T12:49:00.805376+0000","last_peered":"2024-04-28T12:49:00.805376+0000","last_clean":"2024-04-28T12:49:00.805376+0000","last_became_active":"2024-04-28T12:48:59.197088+0000","last_became_peered":"2024-04-28T12:48:59.197088+0000","last_unstale":"2024-04-28T12:49:00.805376+0000","last_undegraded":"2024-04-28T12:49:00.805376+0000","last_fullsized":"2024-04-28T12:49:00.805376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:53:33.391084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.029935+0000","last_change":"2024-04-28T12:48:54.130359+0000","last_active":"2024-04-28T12:49:29.029935+0000","last_peered":"2024-04-28T12:49:29.029935+0000","last_clean":"2024-04-28T12:49:29.029935+0000","last_became_active":"2024-04-28T12:48:54.130043+0000","last_became_peered":"2024-04-28T12:48:54.130043+0000","last_unstale":"2024-04-28T12:49:29.029935+0000","last_undegraded":"2024-04-28T12:49:29.029935+0000","last_fullsized":"2024-04-28T12:49:29.029935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:05:41.353359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800613+0000","last_change":"2024-04-28T12:48:54.960796+0000","last_active":"2024-04-28T12:49:00.800613+0000","last_peered":"2024-04-28T12:49:00.800613+0000","last_clean":"2024-04-28T12:49:00.800613+0000","last_became_active":"2024-04-28T12:48:54.960457+0000","last_became_peered":"2024-04-28T12:48:54.960457+0000","last_unstale":"2024-04-28T12:49:00.800613+0000","last_undegraded":"2024-04-28T12:49:00.800613+0000","last_fullsized":"2024-04-28T12:49:00.800613+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:42:04.096242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800594+0000","last_change":"2024-04-28T12:48:59.193963+0000","last_active":"2024-04-28T12:49:00.800594+0000","last_peered":"2024-04-28T12:49:00.800594+0000","last_clean":"2024-04-28T12:49:00.800594+0000","last_became_active":"2024-04-28T12:48:59.193551+0000","last_became_peered":"2024-04-28T12:48:59.193551+0000","last_unstale":"2024-04-28T12:49:00.800594+0000","last_undegraded":"2024-04-28T12:49:00.800594+0000","last_fullsized":"2024-04-28T12:49:00.800594+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:24:29.474296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:27.991325+0000","last_change":"2024-04-28T12:48:54.109277+0000","last_active":"2024-04-28T12:49:27.991325+0000","last_peered":"2024-04-28T12:49:27.991325+0000","last_clean":"2024-04-28T12:49:27.991325+0000","last_became_active":"2024-04-28T12:48:54.108931+0000","last_became_peered":"2024-04-28T12:48:54.108931+0000","last_unstale":"2024-04-28T12:49:27.991325+0000","last_undegraded":"2024-04-28T12:49:27.991325+0000","last_fullsized":"2024-04-28T12:49:27.991325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:56:29.683311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801688+0000","last_change":"2024-04-28T12:48:54.961014+0000","last_active":"2024-04-28T12:49:00.801688+0000","last_peered":"2024-04-28T12:49:00.801688+0000","last_clean":"2024-04-28T12:49:00.801688+0000","last_became_active":"2024-04-28T12:48:54.960648+0000","last_became_peered":"2024-04-28T12:48:54.960648+0000","last_unstale":"2024-04-28T12:49:00.801688+0000","last_undegraded":"2024-04-28T12:49:00.801688+0000","last_fullsized":"2024-04-28T12:49:00.801688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:02:42.794451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172996+0000","last_change":"2024-04-28T12:48:59.185458+0000","last_active":"2024-04-28T12:49:00.172996+0000","last_peered":"2024-04-28T12:49:00.172996+0000","last_clean":"2024-04-28T12:49:00.172996+0000","last_became_active":"2024-04-28T12:48:59.184959+0000","last_became_peered":"2024-04-28T12:48:59.184959+0000","last_unstale":"2024-04-28T12:49:00.172996+0000","last_undegraded":"2024-04-28T12:49:00.172996+0000","last_fullsized":"2024-04-28T12:49:00.172996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:46:45.723374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804442+0000","last_change":"2024-04-28T12:48:54.134303+0000","last_active":"2024-04-28T12:49:00.804442+0000","last_peered":"2024-04-28T12:49:00.804442+0000","last_clean":"2024-04-28T12:49:00.804442+0000","last_became_active":"2024-04-28T12:48:54.133728+0000","last_became_peered":"2024-04-28T12:48:54.133728+0000","last_unstale":"2024-04-28T12:49:00.804442+0000","last_undegraded":"2024-04-28T12:49:00.804442+0000","last_fullsized":"2024-04-28T12:49:00.804442+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:44:16.444168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801555+0000","last_change":"2024-04-28T12:48:54.983312+0000","last_active":"2024-04-28T12:49:00.801555+0000","last_peered":"2024-04-28T12:49:00.801555+0000","last_clean":"2024-04-28T12:49:00.801555+0000","last_became_active":"2024-04-28T12:48:54.982928+0000","last_became_peered":"2024-04-28T12:48:54.982928+0000","last_unstale":"2024-04-28T12:49:00.801555+0000","last_undegraded":"2024-04-28T12:49:00.801555+0000","last_fullsized":"2024-04-28T12:49:00.801555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:18:22.411048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805788+0000","last_change":"2024-04-28T12:48:59.187641+0000","last_active":"2024-04-28T12:49:00.805788+0000","last_peered":"2024-04-28T12:49:00.805788+0000","last_clean":"2024-04-28T12:49:00.805788+0000","last_became_active":"2024-04-28T12:48:59.186003+0000","last_became_peered":"2024-04-28T12:48:59.186003+0000","last_unstale":"2024-04-28T12:49:00.805788+0000","last_undegraded":"2024-04-28T12:49:00.805788+0000","last_fullsized":"2024-04-28T12:49:00.805788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:50:34.968976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:30.036912+0000","last_change":"2024-04-28T12:48:54.122909+0000","last_active":"2024-04-28T12:49:30.036912+0000","last_peered":"2024-04-28T12:49:30.036912+0000","last_clean":"2024-04-28T12:49:30.036912+0000","last_became_active":"2024-04-28T12:48:54.122661+0000","last_became_peered":"2024-04-28T12:48:54.122661+0000","last_unstale":"2024-04-28T12:49:30.036912+0000","last_undegraded":"2024-04-28T12:49:30.036912+0000","last_fullsized":"2024-04-28T12:49:30.036912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:43:06.119038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800924+0000","last_change":"2024-04-28T12:48:54.956618+0000","last_active":"2024-04-28T12:49:00.800924+0000","last_peered":"2024-04-28T12:49:00.800924+0000","last_clean":"2024-04-28T12:49:00.800924+0000","last_became_active":"2024-04-28T12:48:54.956227+0000","last_became_peered":"2024-04-28T12:48:54.956227+0000","last_unstale":"2024-04-28T12:49:00.800924+0000","last_undegraded":"2024-04-28T12:49:00.800924+0000","last_fullsized":"2024-04-28T12:49:00.800924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:38:02.820771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806512+0000","last_change":"2024-04-28T12:48:59.187017+0000","last_active":"2024-04-28T12:49:00.806512+0000","last_peered":"2024-04-28T12:49:00.806512+0000","last_clean":"2024-04-28T12:49:00.806512+0000","last_became_active":"2024-04-28T12:48:59.186623+0000","last_became_peered":"2024-04-28T12:48:59.186623+0000","last_unstale":"2024-04-28T12:49:00.806512+0000","last_undegraded":"2024-04-28T12:49:00.806512+0000","last_fullsized":"2024-04-28T12:49:00.806512+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:26:53.631400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800721+0000","last_change":"2024-04-28T12:48:54.957348+0000","last_active":"2024-04-28T12:49:00.800721+0000","last_peered":"2024-04-28T12:49:00.800721+0000","last_clean":"2024-04-28T12:49:00.800721+0000","last_became_active":"2024-04-28T12:48:54.956890+0000","last_became_peered":"2024-04-28T12:48:54.956890+0000","last_unstale":"2024-04-28T12:49:00.800721+0000","last_undegraded":"2024-04-28T12:49:00.800721+0000","last_fullsized":"2024-04-28T12:49:00.800721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:41:24.550175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:27.001889+0000","last_change":"2024-04-28T12:48:54.133596+0000","last_active":"2024-04-28T12:49:27.001889+0000","last_peered":"2024-04-28T12:49:27.001889+0000","last_clean":"2024-04-28T12:49:27.001889+0000","last_became_active":"2024-04-28T12:48:54.131632+0000","last_became_peered":"2024-04-28T12:48:54.131632+0000","last_unstale":"2024-04-28T12:49:27.001889+0000","last_undegraded":"2024-04-28T12:49:27.001889+0000","last_fullsized":"2024-04-28T12:49:27.001889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:42:39.352397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173467+0000","last_change":"2024-04-28T12:48:59.174117+0000","last_active":"2024-04-28T12:49:00.173467+0000","last_peered":"2024-04-28T12:49:00.173467+0000","last_clean":"2024-04-28T12:49:00.173467+0000","last_became_active":"2024-04-28T12:48:59.173699+0000","last_became_peered":"2024-04-28T12:48:59.173699+0000","last_unstale":"2024-04-28T12:49:00.173467+0000","last_undegraded":"2024-04-28T12:49:00.173467+0000","last_fullsized":"2024-04-28T12:49:00.173467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:41:51.186413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805626+0000","last_change":"2024-04-28T12:48:54.978658+0000","last_active":"2024-04-28T12:49:00.805626+0000","last_peered":"2024-04-28T12:49:00.805626+0000","last_clean":"2024-04-28T12:49:00.805626+0000","last_became_active":"2024-04-28T12:48:54.978113+0000","last_became_peered":"2024-04-28T12:48:54.978113+0000","last_unstale":"2024-04-28T12:49:00.805626+0000","last_undegraded":"2024-04-28T12:49:00.805626+0000","last_fullsized":"2024-04-28T12:49:00.805626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:33:50.716489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.007267+0000","last_change":"2024-04-28T12:48:54.130301+0000","last_active":"2024-04-28T12:49:28.007267+0000","last_peered":"2024-04-28T12:49:28.007267+0000","last_clean":"2024-04-28T12:49:28.007267+0000","last_became_active":"2024-04-28T12:48:54.130027+0000","last_became_peered":"2024-04-28T12:48:54.130027+0000","last_unstale":"2024-04-28T12:49:28.007267+0000","last_undegraded":"2024-04-28T12:49:28.007267+0000","last_fullsized":"2024-04-28T12:49:28.007267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:30:03.883731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":380,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172744+0000","last_change":"2024-04-28T12:47:34.946674+0000","last_active":"2024-04-28T12:49:00.172744+0000","last_peered":"2024-04-28T12:49:00.172744+0000","last_clean":"2024-04-28T12:49:00.172744+0000","last_became_active":"2024-04-28T12:47:34.946149+0000","last_became_peered":"2024-04-28T12:47:34.946149+0000","last_unstale":"2024-04-28T12:49:00.172744+0000","last_undegraded":"2024-04-28T12:49:00.172744+0000","last_fullsized":"2024-04-28T12:49:00.172744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:43:40.694612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:43:40.694612+0000","last_clean_scrub_stamp":"2024-04-28T12:43:40.694612+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:25:44.776805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800634+0000","last_change":"2024-04-28T12:48:59.190783+0000","last_active":"2024-04-28T12:49:00.800634+0000","last_peered":"2024-04-28T12:49:00.800634+0000","last_clean":"2024-04-28T12:49:00.800634+0000","last_became_active":"2024-04-28T12:48:59.190383+0000","last_became_peered":"2024-04-28T12:48:59.190383+0000","last_unstale":"2024-04-28T12:49:00.800634+0000","last_undegraded":"2024-04-28T12:49:00.800634+0000","last_fullsized":"2024-04-28T12:49:00.800634+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:39:41.711227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802259+0000","last_change":"2024-04-28T12:48:54.979911+0000","last_active":"2024-04-28T12:49:00.802259+0000","last_peered":"2024-04-28T12:49:00.802259+0000","last_clean":"2024-04-28T12:49:00.802259+0000","last_became_active":"2024-04-28T12:48:54.974708+0000","last_became_peered":"2024-04-28T12:48:54.974708+0000","last_unstale":"2024-04-28T12:49:00.802259+0000","last_undegraded":"2024-04-28T12:49:00.802259+0000","last_fullsized":"2024-04-28T12:49:00.802259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:48:54.919849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:26.996348+0000","last_change":"2024-04-28T12:48:54.116384+0000","last_active":"2024-04-28T12:49:26.996348+0000","last_peered":"2024-04-28T12:49:26.996348+0000","last_clean":"2024-04-28T12:49:26.996348+0000","last_became_active":"2024-04-28T12:48:54.114362+0000","last_became_peered":"2024-04-28T12:48:54.114362+0000","last_unstale":"2024-04-28T12:49:26.996348+0000","last_undegraded":"2024-04-28T12:49:26.996348+0000","last_fullsized":"2024-04-28T12:49:26.996348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:20.406641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801295+0000","last_change":"2024-04-28T12:48:59.181370+0000","last_active":"2024-04-28T12:49:00.801295+0000","last_peered":"2024-04-28T12:49:00.801295+0000","last_clean":"2024-04-28T12:49:00.801295+0000","last_became_active":"2024-04-28T12:48:59.180852+0000","last_became_peered":"2024-04-28T12:48:59.180852+0000","last_unstale":"2024-04-28T12:49:00.801295+0000","last_undegraded":"2024-04-28T12:49:00.801295+0000","last_fullsized":"2024-04-28T12:49:00.801295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:08:55.532273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.033551+0000","last_change":"2024-04-28T12:48:54.129792+0000","last_active":"2024-04-28T12:49:29.033551+0000","last_peered":"2024-04-28T12:49:29.033551+0000","last_clean":"2024-04-28T12:49:29.033551+0000","last_became_active":"2024-04-28T12:48:54.129437+0000","last_became_peered":"2024-04-28T12:48:54.129437+0000","last_unstale":"2024-04-28T12:49:29.033551+0000","last_undegraded":"2024-04-28T12:49:29.033551+0000","last_fullsized":"2024-04-28T12:49:29.033551+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:22:14.658847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804735+0000","last_change":"2024-04-28T12:48:54.980482+0000","last_active":"2024-04-28T12:49:00.804735+0000","last_peered":"2024-04-28T12:49:00.804735+0000","last_clean":"2024-04-28T12:49:00.804735+0000","last_became_active":"2024-04-28T12:48:54.980022+0000","last_became_peered":"2024-04-28T12:48:54.980022+0000","last_unstale":"2024-04-28T12:49:00.804735+0000","last_undegraded":"2024-04-28T12:49:00.804735+0000","last_fullsized":"2024-04-28T12:49:00.804735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:50:11.573598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.803612+0000","last_change":"2024-04-28T12:48:59.188749+0000","last_active":"2024-04-28T12:49:00.803612+0000","last_peered":"2024-04-28T12:49:00.803612+0000","last_clean":"2024-04-28T12:49:00.803612+0000","last_became_active":"2024-04-28T12:48:59.187735+0000","last_became_peered":"2024-04-28T12:48:59.187735+0000","last_unstale":"2024-04-28T12:49:00.803612+0000","last_undegraded":"2024-04-28T12:49:00.803612+0000","last_fullsized":"2024-04-28T12:49:00.803612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:50:42.948378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.009741+0000","last_change":"2024-04-28T12:48:54.132690+0000","last_active":"2024-04-28T12:49:28.009741+0000","last_peered":"2024-04-28T12:49:28.009741+0000","last_clean":"2024-04-28T12:49:28.009741+0000","last_became_active":"2024-04-28T12:48:54.132300+0000","last_became_peered":"2024-04-28T12:48:54.132300+0000","last_unstale":"2024-04-28T12:49:28.009741+0000","last_undegraded":"2024-04-28T12:49:28.009741+0000","last_fullsized":"2024-04-28T12:49:28.009741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:14.194355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.807793+0000","last_change":"2024-04-28T12:48:54.981277+0000","last_active":"2024-04-28T12:49:00.807793+0000","last_peered":"2024-04-28T12:49:00.807793+0000","last_clean":"2024-04-28T12:49:00.807793+0000","last_became_active":"2024-04-28T12:48:54.977366+0000","last_became_peered":"2024-04-28T12:48:54.977366+0000","last_unstale":"2024-04-28T12:49:00.807793+0000","last_undegraded":"2024-04-28T12:49:00.807793+0000","last_fullsized":"2024-04-28T12:49:00.807793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:25.323001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172969+0000","last_change":"2024-04-28T12:48:59.174010+0000","last_active":"2024-04-28T12:49:00.172969+0000","last_peered":"2024-04-28T12:49:00.172969+0000","last_clean":"2024-04-28T12:49:00.172969+0000","last_became_active":"2024-04-28T12:48:59.173370+0000","last_became_peered":"2024-04-28T12:48:59.173370+0000","last_unstale":"2024-04-28T12:49:00.172969+0000","last_undegraded":"2024-04-28T12:49:00.172969+0000","last_fullsized":"2024-04-28T12:49:00.172969+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:07:43.258318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:29.031617+0000","last_change":"2024-04-28T12:48:54.127134+0000","last_active":"2024-04-28T12:49:29.031617+0000","last_peered":"2024-04-28T12:49:29.031617+0000","last_clean":"2024-04-28T12:49:29.031617+0000","last_became_active":"2024-04-28T12:48:54.126845+0000","last_became_peered":"2024-04-28T12:48:54.126845+0000","last_unstale":"2024-04-28T12:49:29.031617+0000","last_undegraded":"2024-04-28T12:49:29.031617+0000","last_fullsized":"2024-04-28T12:49:29.031617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:43.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.178342+0000","last_change":"2024-04-28T12:48:54.986671+0000","last_active":"2024-04-28T12:49:00.178342+0000","last_peered":"2024-04-28T12:49:00.178342+0000","last_clean":"2024-04-28T12:49:00.178342+0000","last_became_active":"2024-04-28T12:48:54.985410+0000","last_became_peered":"2024-04-28T12:48:54.985410+0000","last_unstale":"2024-04-28T12:49:00.178342+0000","last_undegraded":"2024-04-28T12:49:00.178342+0000","last_fullsized":"2024-04-28T12:49:00.178342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:15:56.354272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801131+0000","last_change":"2024-04-28T12:48:59.188456+0000","last_active":"2024-04-28T12:49:00.801131+0000","last_peered":"2024-04-28T12:49:00.801131+0000","last_clean":"2024-04-28T12:49:00.801131+0000","last_became_active":"2024-04-28T12:48:59.187984+0000","last_became_peered":"2024-04-28T12:48:59.187984+0000","last_unstale":"2024-04-28T12:49:00.801131+0000","last_undegraded":"2024-04-28T12:49:00.801131+0000","last_fullsized":"2024-04-28T12:49:00.801131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:38:32.653563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.983690+0000","last_change":"2024-04-28T12:48:54.125483+0000","last_active":"2024-04-28T12:49:25.983690+0000","last_peered":"2024-04-28T12:49:25.983690+0000","last_clean":"2024-04-28T12:49:25.983690+0000","last_became_active":"2024-04-28T12:48:54.125261+0000","last_became_peered":"2024-04-28T12:48:54.125261+0000","last_unstale":"2024-04-28T12:49:25.983690+0000","last_undegraded":"2024-04-28T12:49:25.983690+0000","last_fullsized":"2024-04-28T12:49:25.983690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:32:24.986128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172700+0000","last_change":"2024-04-28T12:48:54.960662+0000","last_active":"2024-04-28T12:49:00.172700+0000","last_peered":"2024-04-28T12:49:00.172700+0000","last_clean":"2024-04-28T12:49:00.172700+0000","last_became_active":"2024-04-28T12:48:54.960295+0000","last_became_peered":"2024-04-28T12:48:54.960295+0000","last_unstale":"2024-04-28T12:49:00.172700+0000","last_undegraded":"2024-04-28T12:49:00.172700+0000","last_fullsized":"2024-04-28T12:49:00.172700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:26.190901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806754+0000","last_change":"2024-04-28T12:48:59.194137+0000","last_active":"2024-04-28T12:49:00.806754+0000","last_peered":"2024-04-28T12:49:00.806754+0000","last_clean":"2024-04-28T12:49:00.806754+0000","last_became_active":"2024-04-28T12:48:59.193740+0000","last_became_peered":"2024-04-28T12:48:59.193740+0000","last_unstale":"2024-04-28T12:49:00.806754+0000","last_undegraded":"2024-04-28T12:49:00.806754+0000","last_fullsized":"2024-04-28T12:49:00.806754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:58:52.962196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.984733+0000","last_change":"2024-04-28T12:48:54.140883+0000","last_active":"2024-04-28T12:49:25.984733+0000","last_peered":"2024-04-28T12:49:25.984733+0000","last_clean":"2024-04-28T12:49:25.984733+0000","last_became_active":"2024-04-28T12:48:54.140548+0000","last_became_peered":"2024-04-28T12:48:54.140548+0000","last_unstale":"2024-04-28T12:49:25.984733+0000","last_undegraded":"2024-04-28T12:49:25.984733+0000","last_fullsized":"2024-04-28T12:49:25.984733+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:31.465266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800767+0000","last_change":"2024-04-28T12:48:54.976359+0000","last_active":"2024-04-28T12:49:00.800767+0000","last_peered":"2024-04-28T12:49:00.800767+0000","last_clean":"2024-04-28T12:49:00.800767+0000","last_became_active":"2024-04-28T12:48:54.976028+0000","last_became_peered":"2024-04-28T12:48:54.976028+0000","last_unstale":"2024-04-28T12:49:00.800767+0000","last_undegraded":"2024-04-28T12:49:00.800767+0000","last_fullsized":"2024-04-28T12:49:00.800767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:02:48.498276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802625+0000","last_change":"2024-04-28T12:48:59.179755+0000","last_active":"2024-04-28T12:49:00.802625+0000","last_peered":"2024-04-28T12:49:00.802625+0000","last_clean":"2024-04-28T12:49:00.802625+0000","last_became_active":"2024-04-28T12:48:59.179258+0000","last_became_peered":"2024-04-28T12:48:59.179258+0000","last_unstale":"2024-04-28T12:49:00.802625+0000","last_undegraded":"2024-04-28T12:49:00.802625+0000","last_fullsized":"2024-04-28T12:49:00.802625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:59:38.863875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.009580+0000","last_change":"2024-04-28T12:48:54.121894+0000","last_active":"2024-04-28T12:49:28.009580+0000","last_peered":"2024-04-28T12:49:28.009580+0000","last_clean":"2024-04-28T12:49:28.009580+0000","last_became_active":"2024-04-28T12:48:54.121605+0000","last_became_peered":"2024-04-28T12:48:54.121605+0000","last_unstale":"2024-04-28T12:49:28.009580+0000","last_undegraded":"2024-04-28T12:49:28.009580+0000","last_fullsized":"2024-04-28T12:49:28.009580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:55:11.181943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.806914+0000","last_change":"2024-04-28T12:48:54.990809+0000","last_active":"2024-04-28T12:49:00.806914+0000","last_peered":"2024-04-28T12:49:00.806914+0000","last_clean":"2024-04-28T12:49:00.806914+0000","last_became_active":"2024-04-28T12:48:54.990251+0000","last_became_peered":"2024-04-28T12:48:54.990251+0000","last_unstale":"2024-04-28T12:49:00.806914+0000","last_undegraded":"2024-04-28T12:49:00.806914+0000","last_fullsized":"2024-04-28T12:49:00.806914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:14:30.017815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802181+0000","last_change":"2024-04-28T12:48:59.193287+0000","last_active":"2024-04-28T12:49:00.802181+0000","last_peered":"2024-04-28T12:49:00.802181+0000","last_clean":"2024-04-28T12:49:00.802181+0000","last_became_active":"2024-04-28T12:48:59.192755+0000","last_became_peered":"2024-04-28T12:48:59.192755+0000","last_unstale":"2024-04-28T12:49:00.802181+0000","last_undegraded":"2024-04-28T12:49:00.802181+0000","last_fullsized":"2024-04-28T12:49:00.802181+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:00:33.790384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:25.981363+0000","last_change":"2024-04-28T12:48:54.121052+0000","last_active":"2024-04-28T12:49:25.981363+0000","last_peered":"2024-04-28T12:49:25.981363+0000","last_clean":"2024-04-28T12:49:25.981363+0000","last_became_active":"2024-04-28T12:48:54.120702+0000","last_became_peered":"2024-04-28T12:48:54.120702+0000","last_unstale":"2024-04-28T12:49:25.981363+0000","last_undegraded":"2024-04-28T12:49:25.981363+0000","last_fullsized":"2024-04-28T12:49:25.981363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:20:45.771466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.172991+0000","last_change":"2024-04-28T12:48:54.970270+0000","last_active":"2024-04-28T12:49:00.172991+0000","last_peered":"2024-04-28T12:49:00.172991+0000","last_clean":"2024-04-28T12:49:00.172991+0000","last_became_active":"2024-04-28T12:48:54.970010+0000","last_became_peered":"2024-04-28T12:48:54.970010+0000","last_unstale":"2024-04-28T12:49:00.172991+0000","last_undegraded":"2024-04-28T12:49:00.172991+0000","last_fullsized":"2024-04-28T12:49:00.172991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:05:03.578751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800870+0000","last_change":"2024-04-28T12:48:59.186197+0000","last_active":"2024-04-28T12:49:00.800870+0000","last_peered":"2024-04-28T12:49:00.800870+0000","last_clean":"2024-04-28T12:49:00.800870+0000","last_became_active":"2024-04-28T12:48:59.185749+0000","last_became_peered":"2024-04-28T12:48:59.185749+0000","last_unstale":"2024-04-28T12:49:00.800870+0000","last_undegraded":"2024-04-28T12:49:00.800870+0000","last_fullsized":"2024-04-28T12:49:00.800870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:36.336775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:30.035394+0000","last_change":"2024-04-28T12:48:54.118725+0000","last_active":"2024-04-28T12:49:30.035394+0000","last_peered":"2024-04-28T12:49:30.035394+0000","last_clean":"2024-04-28T12:49:30.035394+0000","last_became_active":"2024-04-28T12:48:54.118419+0000","last_became_peered":"2024-04-28T12:48:54.118419+0000","last_unstale":"2024-04-28T12:49:30.035394+0000","last_undegraded":"2024-04-28T12:49:30.035394+0000","last_fullsized":"2024-04-28T12:49:30.035394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:25:17.959905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173521+0000","last_change":"2024-04-28T12:48:54.959518+0000","last_active":"2024-04-28T12:49:00.173521+0000","last_peered":"2024-04-28T12:49:00.173521+0000","last_clean":"2024-04-28T12:49:00.173521+0000","last_became_active":"2024-04-28T12:48:54.959101+0000","last_became_peered":"2024-04-28T12:48:54.959101+0000","last_unstale":"2024-04-28T12:49:00.173521+0000","last_undegraded":"2024-04-28T12:49:00.173521+0000","last_fullsized":"2024-04-28T12:49:00.173521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:38:50.496961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177195+0000","last_change":"2024-04-28T12:48:59.180941+0000","last_active":"2024-04-28T12:49:00.177195+0000","last_peered":"2024-04-28T12:49:00.177195+0000","last_clean":"2024-04-28T12:49:00.177195+0000","last_became_active":"2024-04-28T12:48:59.176889+0000","last_became_peered":"2024-04-28T12:48:59.176889+0000","last_unstale":"2024-04-28T12:49:00.177195+0000","last_undegraded":"2024-04-28T12:49:00.177195+0000","last_fullsized":"2024-04-28T12:49:00.177195+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:28:21.853493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:28.009956+0000","last_change":"2024-04-28T12:48:54.126265+0000","last_active":"2024-04-28T12:49:28.009956+0000","last_peered":"2024-04-28T12:49:28.009956+0000","last_clean":"2024-04-28T12:49:28.009956+0000","last_became_active":"2024-04-28T12:48:54.125918+0000","last_became_peered":"2024-04-28T12:48:54.125918+0000","last_unstale":"2024-04-28T12:49:28.009956+0000","last_undegraded":"2024-04-28T12:49:28.009956+0000","last_fullsized":"2024-04-28T12:49:28.009956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:24:26.340143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173220+0000","last_change":"2024-04-28T12:48:54.985335+0000","last_active":"2024-04-28T12:49:00.173220+0000","last_peered":"2024-04-28T12:49:00.173220+0000","last_clean":"2024-04-28T12:49:00.173220+0000","last_became_active":"2024-04-28T12:48:54.985017+0000","last_became_peered":"2024-04-28T12:48:54.985017+0000","last_unstale":"2024-04-28T12:49:00.173220+0000","last_undegraded":"2024-04-28T12:49:00.173220+0000","last_fullsized":"2024-04-28T12:49:00.173220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:49:21.821976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800227+0000","last_change":"2024-04-28T12:48:59.176438+0000","last_active":"2024-04-28T12:49:00.800227+0000","last_peered":"2024-04-28T12:49:00.800227+0000","last_clean":"2024-04-28T12:49:00.800227+0000","last_became_active":"2024-04-28T12:48:59.175773+0000","last_became_peered":"2024-04-28T12:48:59.175773+0000","last_unstale":"2024-04-28T12:49:00.800227+0000","last_undegraded":"2024-04-28T12:49:00.800227+0000","last_fullsized":"2024-04-28T12:49:00.800227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:32.687553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.805149+0000","last_change":"2024-04-28T12:48:54.130452+0000","last_active":"2024-04-28T12:49:00.805149+0000","last_peered":"2024-04-28T12:49:00.805149+0000","last_clean":"2024-04-28T12:49:00.805149+0000","last_became_active":"2024-04-28T12:48:54.130090+0000","last_became_peered":"2024-04-28T12:48:54.130090+0000","last_unstale":"2024-04-28T12:49:00.805149+0000","last_undegraded":"2024-04-28T12:49:00.805149+0000","last_fullsized":"2024-04-28T12:49:00.805149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:02:34.014710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173492+0000","last_change":"2024-04-28T12:48:54.961838+0000","last_active":"2024-04-28T12:49:00.173492+0000","last_peered":"2024-04-28T12:49:00.173492+0000","last_clean":"2024-04-28T12:49:00.173492+0000","last_became_active":"2024-04-28T12:48:54.961525+0000","last_became_peered":"2024-04-28T12:48:54.961525+0000","last_unstale":"2024-04-28T12:49:00.173492+0000","last_undegraded":"2024-04-28T12:49:00.173492+0000","last_fullsized":"2024-04-28T12:49:00.173492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:06:46.077023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800883+0000","last_change":"2024-04-28T12:48:54.959806+0000","last_active":"2024-04-28T12:49:00.800883+0000","last_peered":"2024-04-28T12:49:00.800883+0000","last_clean":"2024-04-28T12:49:00.800883+0000","last_became_active":"2024-04-28T12:48:54.959491+0000","last_became_peered":"2024-04-28T12:48:54.959491+0000","last_unstale":"2024-04-28T12:49:00.800883+0000","last_undegraded":"2024-04-28T12:49:00.800883+0000","last_fullsized":"2024-04-28T12:49:00.800883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:12:25.367185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.174176+0000","last_change":"2024-04-28T12:48:54.960035+0000","last_active":"2024-04-28T12:49:00.174176+0000","last_peered":"2024-04-28T12:49:00.174176+0000","last_clean":"2024-04-28T12:49:00.174176+0000","last_became_active":"2024-04-28T12:48:54.959684+0000","last_became_peered":"2024-04-28T12:48:54.959684+0000","last_unstale":"2024-04-28T12:49:00.174176+0000","last_undegraded":"2024-04-28T12:49:00.174176+0000","last_fullsized":"2024-04-28T12:49:00.174176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:46:36.484928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173089+0000","last_change":"2024-04-28T12:48:54.984682+0000","last_active":"2024-04-28T12:49:00.173089+0000","last_peered":"2024-04-28T12:49:00.173089+0000","last_clean":"2024-04-28T12:49:00.173089+0000","last_became_active":"2024-04-28T12:48:54.984346+0000","last_became_peered":"2024-04-28T12:48:54.984346+0000","last_unstale":"2024-04-28T12:49:00.173089+0000","last_undegraded":"2024-04-28T12:49:00.173089+0000","last_fullsized":"2024-04-28T12:49:00.173089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:22:57.828641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.177475+0000","last_change":"2024-04-28T12:48:54.981526+0000","last_active":"2024-04-28T12:49:00.177475+0000","last_peered":"2024-04-28T12:49:00.177475+0000","last_clean":"2024-04-28T12:49:00.177475+0000","last_became_active":"2024-04-28T12:48:54.981212+0000","last_became_peered":"2024-04-28T12:48:54.981212+0000","last_unstale":"2024-04-28T12:49:00.177475+0000","last_undegraded":"2024-04-28T12:49:00.177475+0000","last_fullsized":"2024-04-28T12:49:00.177475+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:29:00.976903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804836+0000","last_change":"2024-04-28T12:48:54.986664+0000","last_active":"2024-04-28T12:49:00.804836+0000","last_peered":"2024-04-28T12:49:00.804836+0000","last_clean":"2024-04-28T12:49:00.804836+0000","last_became_active":"2024-04-28T12:48:54.986358+0000","last_became_peered":"2024-04-28T12:48:54.986358+0000","last_unstale":"2024-04-28T12:49:00.804836+0000","last_undegraded":"2024-04-28T12:49:00.804836+0000","last_fullsized":"2024-04-28T12:49:00.804836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:11:08.970414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801788+0000","last_change":"2024-04-28T12:48:54.964342+0000","last_active":"2024-04-28T12:49:00.801788+0000","last_peered":"2024-04-28T12:49:00.801788+0000","last_clean":"2024-04-28T12:49:00.801788+0000","last_became_active":"2024-04-28T12:48:54.963980+0000","last_became_peered":"2024-04-28T12:48:54.963980+0000","last_unstale":"2024-04-28T12:49:00.801788+0000","last_undegraded":"2024-04-28T12:49:00.801788+0000","last_fullsized":"2024-04-28T12:49:00.801788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:17:07.772838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.173837+0000","last_change":"2024-04-28T12:48:54.963955+0000","last_active":"2024-04-28T12:49:00.173837+0000","last_peered":"2024-04-28T12:49:00.173837+0000","last_clean":"2024-04-28T12:49:00.173837+0000","last_became_active":"2024-04-28T12:48:54.963572+0000","last_became_peered":"2024-04-28T12:48:54.963572+0000","last_unstale":"2024-04-28T12:49:00.173837+0000","last_undegraded":"2024-04-28T12:49:00.173837+0000","last_fullsized":"2024-04-28T12:49:00.173837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:18.077455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.807988+0000","last_change":"2024-04-28T12:48:54.978143+0000","last_active":"2024-04-28T12:49:00.807988+0000","last_peered":"2024-04-28T12:49:00.807988+0000","last_clean":"2024-04-28T12:49:00.807988+0000","last_became_active":"2024-04-28T12:48:54.977715+0000","last_became_peered":"2024-04-28T12:48:54.977715+0000","last_unstale":"2024-04-28T12:49:00.807988+0000","last_undegraded":"2024-04-28T12:49:00.807988+0000","last_fullsized":"2024-04-28T12:49:00.807988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:47:17.071081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.800439+0000","last_change":"2024-04-28T12:48:54.982613+0000","last_active":"2024-04-28T12:49:00.800439+0000","last_peered":"2024-04-28T12:49:00.800439+0000","last_clean":"2024-04-28T12:49:00.800439+0000","last_became_active":"2024-04-28T12:48:54.982231+0000","last_became_peered":"2024-04-28T12:48:54.982231+0000","last_unstale":"2024-04-28T12:49:00.800439+0000","last_undegraded":"2024-04-28T12:49:00.800439+0000","last_fullsized":"2024-04-28T12:49:00.800439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:22.206377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.804186+0000","last_change":"2024-04-28T12:48:54.978398+0000","last_active":"2024-04-28T12:49:00.804186+0000","last_peered":"2024-04-28T12:49:00.804186+0000","last_clean":"2024-04-28T12:49:00.804186+0000","last_became_active":"2024-04-28T12:48:54.977015+0000","last_became_peered":"2024-04-28T12:48:54.977015+0000","last_unstale":"2024-04-28T12:49:00.804186+0000","last_undegraded":"2024-04-28T12:49:00.804186+0000","last_fullsized":"2024-04-28T12:49:00.804186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:39:39.499850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801910+0000","last_change":"2024-04-28T12:48:54.982870+0000","last_active":"2024-04-28T12:49:00.801910+0000","last_peered":"2024-04-28T12:49:00.801910+0000","last_clean":"2024-04-28T12:49:00.801910+0000","last_became_active":"2024-04-28T12:48:54.982308+0000","last_became_peered":"2024-04-28T12:48:54.982308+0000","last_unstale":"2024-04-28T12:49:00.801910+0000","last_undegraded":"2024-04-28T12:49:00.801910+0000","last_fullsized":"2024-04-28T12:49:00.801910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:47:45.976825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.802980+0000","last_change":"2024-04-28T12:48:54.987904+0000","last_active":"2024-04-28T12:49:00.802980+0000","last_peered":"2024-04-28T12:49:00.802980+0000","last_clean":"2024-04-28T12:49:00.802980+0000","last_became_active":"2024-04-28T12:48:54.987551+0000","last_became_peered":"2024-04-28T12:48:54.987551+0000","last_unstale":"2024-04-28T12:49:00.802980+0000","last_undegraded":"2024-04-28T12:49:00.802980+0000","last_fullsized":"2024-04-28T12:49:00.802980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:30:36.831005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-28T12:49:00.801542+0000","last_change":"2024-04-28T12:48:54.963176+0000","last_active":"2024-04-28T12:49:00.801542+0000","last_peered":"2024-04-28T12:49:00.801542+0000","last_clean":"2024-04-28T12:49:00.801542+0000","last_became_active":"2024-04-28T12:48:54.962830+0000","last_became_peered":"2024-04-28T12:48:54.962830+0000","last_unstale":"2024-04-28T12:49:00.801542+0000","last_undegraded":"2024-04-28T12:49:00.801542+0000","last_fullsized":"2024-04-28T12:49:00.801542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:10:40.671376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47954,"num_objects":91,"num_object_clones":0,"num_object_copies":273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":0,"num_read":26,"num_read_kb":59,"num_write":210,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":61,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":142815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383700,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":477756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645337,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1654784,"data_stored":1092810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808862,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":477992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":478674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332140,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1613824,"data_stored":1075562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":477264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":502067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":479743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":503069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182465,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":477368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":477840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36916,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708236,"statfs":{"total":95995035648,"available":95957233664,"internally_reserved":0,"allocated":1617920,"data_stored":1069780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":42598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:49:32.926 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-28T12:49:32.927 INFO:tasks.cephadm:Setting up client nodes... 2024-04-28T12:49:32.927 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-28T12:49:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:33.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:33 smithi171 ceph-mon[63127]: pgmap v337: 129 pgs: 129 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-28T12:49:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:33 smithi129 ceph-mon[62418]: pgmap v337: 129 pgs: 129 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-28T12:49:33.552 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:33 smithi106 ceph-mon[28081]: pgmap v337: 129 pgs: 129 active+clean; 623 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2024-04-28T12:49:34.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:34 smithi171 ceph-mon[63127]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:49:34.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:34 smithi171 ceph-mon[63127]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:49:34.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:34 smithi171 ceph-mon[63127]: pgmap v338: 129 pgs: 129 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-04-28T12:49:34.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:34 smithi129 ceph-mon[62418]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:49:34.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:34 smithi129 ceph-mon[62418]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:49:34.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:34 smithi129 ceph-mon[62418]: pgmap v338: 129 pgs: 129 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-04-28T12:49:34.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:34 smithi106 ceph-mon[28081]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:49:34.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:34 smithi106 ceph-mon[28081]: from='client.24877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:49:34.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:34 smithi106 ceph-mon[28081]: pgmap v338: 129 pgs: 129 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-04-28T12:49:35.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.715+0000 7fee17800700 1 -- 172.21.15.106:0/4240668204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee101051a0 msgr2=0x7fee1010ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:35.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.715+0000 7fee17800700 1 --2- 172.21.15.106:0/4240668204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee101051a0 0x7fee1010ba50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fee08009f90 tx=0x7fee0800b330 comp rx=0 tx=0).stop 2024-04-28T12:49:35.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.715+0000 7fee17800700 1 -- 172.21.15.106:0/4240668204 shutdown_connections 2024-04-28T12:49:35.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.715+0000 7fee17800700 1 --2- 172.21.15.106:0/4240668204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee101051a0 0x7fee1010ba50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:35.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.715+0000 7fee17800700 1 --2- 172.21.15.106:0/4240668204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee10101780 0x7fee10101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:35.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.715+0000 7fee17800700 1 --2- 172.21.15.106:0/4240668204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10104800 0x7fee10104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:35.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.715+0000 7fee17800700 1 -- 172.21.15.106:0/4240668204 >> 172.21.15.106:0/4240668204 conn(0x7fee100fd0c0 msgr2=0x7fee100ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:35.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.715+0000 7fee17800700 1 -- 172.21.15.106:0/4240668204 shutdown_connections 2024-04-28T12:49:35.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.715+0000 7fee17800700 1 -- 172.21.15.106:0/4240668204 wait complete. 2024-04-28T12:49:35.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.716+0000 7fee17800700 1 Processor -- start 2024-04-28T12:49:35.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.717+0000 7fee17800700 1 -- start start 2024-04-28T12:49:35.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.717+0000 7fee17800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10101780 0x7fee1019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:35.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.717+0000 7fee17800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee10104800 0x7fee1019bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:35.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.717+0000 7fee17800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee101051a0 0x7fee101a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:35.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.717+0000 7fee17800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fee1010ef90 con 0x7fee10104800 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.717+0000 7fee17800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fee1010ee10 con 0x7fee10101780 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.717+0000 7fee17800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fee1010f110 con 0x7fee101051a0 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee14e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10101780 0x7fee1019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee0fe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee10104800 0x7fee1019bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee14e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10101780 0x7fee1019b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:60540/0 (socket says 172.21.15.106:60540) 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee14e00700 1 -- 172.21.15.106:0/631348071 learned_addr learned my addr 172.21.15.106:0/631348071 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee15800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee101051a0 0x7fee101a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee15800700 1 -- 172.21.15.106:0/631348071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10101780 msgr2=0x7fee1019b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee15800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10101780 0x7fee1019b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee15800700 1 -- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee10104800 msgr2=0x7fee1019bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee15800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee10104800 0x7fee1019bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee15800700 1 -- 172.21.15.106:0/631348071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee101a3380 con 0x7fee101051a0 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee14e00700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10101780 0x7fee1019b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:35.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.718+0000 7fee0fe00700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee10104800 0x7fee1019bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:35.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.719+0000 7fee15800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee101051a0 0x7fee101a2ce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fee08009f90 tx=0x7fee080078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:35.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.719+0000 7fee0d600700 1 -- 172.21.15.106:0/631348071 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0801d670 con 0x7fee101051a0 2024-04-28T12:49:35.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.719+0000 7fee0d600700 1 -- 172.21.15.106:0/631348071 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee08022070 con 0x7fee101051a0 2024-04-28T12:49:35.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.719+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee101a3610 con 0x7fee101051a0 2024-04-28T12:49:35.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.719+0000 7fee0d600700 1 -- 172.21.15.106:0/631348071 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0801bac0 con 0x7fee101051a0 2024-04-28T12:49:35.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.719+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee101a3ac0 con 0x7fee101051a0 2024-04-28T12:49:35.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.720+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee10051960 con 0x7fee101051a0 2024-04-28T12:49:35.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.724+0000 7fee0d600700 1 -- 172.21.15.106:0/631348071 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fee08009ae0 con 0x7fee101051a0 2024-04-28T12:49:35.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.725+0000 7fee0d600700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fedf80a3e00 0x7fedf80a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:35.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.726+0000 7fee0d600700 1 -- 172.21.15.106:0/631348071 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fee0800f020 con 0x7fee101051a0 2024-04-28T12:49:35.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.726+0000 7fee14e00700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fedf80a3e00 0x7fedf80a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:35.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.726+0000 7fee14e00700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fedf80a3e00 0x7fedf80a62c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee04005950 tx=0x7fee04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:35.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:35.727+0000 7fee0d600700 1 -- 172.21.15.106:0/631348071 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fee0809df20 con 0x7fee101051a0 2024-04-28T12:49:36.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.048+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fee101a3f80 con 0x7fee101051a0 2024-04-28T12:49:36.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.055+0000 7fee0d600700 1 -- 172.21.15.106:0/631348071 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fee101a3f80 con 0x7fee101051a0 2024-04-28T12:49:36.051 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2024-04-28T12:49:36.051 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDgRS5mC4wGAxAA5jAFldXFAo6VD8eMDJv8Bw== 2024-04-28T12:49:36.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.057+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fedf80a3e00 msgr2=0x7fedf80a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:36.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.057+0000 7fee17800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fedf80a3e00 0x7fedf80a62c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee04005950 tx=0x7fee04009260 comp rx=0 tx=0).stop 2024-04-28T12:49:36.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.057+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee101051a0 msgr2=0x7fee101a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:36.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.057+0000 7fee17800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee101051a0 0x7fee101a2ce0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fee08009f90 tx=0x7fee080078e0 comp rx=0 tx=0).stop 2024-04-28T12:49:36.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.058+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 shutdown_connections 2024-04-28T12:49:36.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.058+0000 7fee17800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fedf80a3e00 0x7fedf80a62c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:36.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.058+0000 7fee17800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee101051a0 0x7fee101a2ce0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:36.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.058+0000 7fee17800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee10101780 0x7fee1019b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:36.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.058+0000 7fee17800700 1 --2- 172.21.15.106:0/631348071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee10104800 0x7fee1019bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:36.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.058+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 >> 172.21.15.106:0/631348071 conn(0x7fee100fd0c0 msgr2=0x7fee101025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:36.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.058+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 shutdown_connections 2024-04-28T12:49:36.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:36.058+0000 7fee17800700 1 -- 172.21.15.106:0/631348071 wait complete. 2024-04-28T12:49:36.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:36 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/631348071' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-28T12:49:36.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:36 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-28T12:49:36.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:36 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-28T12:49:36.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:36 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/631348071' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-28T12:49:36.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:36 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-28T12:49:36.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:36 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-28T12:49:36.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:36 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/631348071' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-28T12:49:36.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:36 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-28T12:49:36.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:36 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-28T12:49:36.886 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:49:36.886 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-28T12:49:36.887 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-28T12:49:36.991 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-28T12:49:36.991 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-28T12:49:36.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-28T12:49:37.448 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:37.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:37 smithi171 ceph-mon[63127]: pgmap v339: 129 pgs: 129 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.1 KiB/s wr, 21 op/s 2024-04-28T12:49:37.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:37 smithi129 ceph-mon[62418]: pgmap v339: 129 pgs: 129 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.1 KiB/s wr, 21 op/s 2024-04-28T12:49:37.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:37 smithi106 ceph-mon[28081]: pgmap v339: 129 pgs: 129 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.1 KiB/s wr, 21 op/s 2024-04-28T12:49:39.395 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:39 smithi106 ceph-mon[28081]: pgmap v340: 129 pgs: 129 active+clean; 629 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.6 KiB/s wr, 23 op/s 2024-04-28T12:49:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:39 smithi171 ceph-mon[63127]: pgmap v340: 129 pgs: 129 active+clean; 629 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.6 KiB/s wr, 23 op/s 2024-04-28T12:49:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:39 smithi129 ceph-mon[62418]: pgmap v340: 129 pgs: 129 active+clean; 629 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.6 KiB/s wr, 23 op/s 2024-04-28T12:49:39.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.565+0000 7f486b600700 1 -- 172.21.15.106:0/175470441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f486410a410 msgr2=0x7f4864116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:39.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.565+0000 7f486b600700 1 --2- 172.21.15.106:0/175470441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f486410a410 0x7f4864116920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f485c009990 tx=0x7f485c009ca0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.565+0000 7f486b600700 1 -- 172.21.15.106:0/175470441 shutdown_connections 2024-04-28T12:49:39.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.565+0000 7f486b600700 1 --2- 172.21.15.106:0/175470441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f486410a410 0x7f4864116920 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.565+0000 7f486b600700 1 --2- 172.21.15.106:0/175470441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f486410fa50 0x7f486410fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.565+0000 7f486b600700 1 --2- 172.21.15.106:0/175470441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4864109a50 0x7f4864109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.565+0000 7f486b600700 1 -- 172.21.15.106:0/175470441 >> 172.21.15.106:0/175470441 conn(0x7f4864076bf0 msgr2=0x7f4864108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:39.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.565+0000 7f486b600700 1 -- 172.21.15.106:0/175470441 shutdown_connections 2024-04-28T12:49:39.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.566+0000 7f486b600700 1 -- 172.21.15.106:0/175470441 wait complete. 2024-04-28T12:49:39.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.566+0000 7f486b600700 1 Processor -- start 2024-04-28T12:49:39.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f486b600700 1 -- start start 2024-04-28T12:49:39.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f486b600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a50 0x7f48641a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:39.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f486b600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f486410a410 0x7f48641a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:39.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f486b600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f486410fa50 0x7f48641a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:39.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f486b600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4864119c90 con 0x7f486410a410 2024-04-28T12:49:39.564 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f486b600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4864119b10 con 0x7f4864109a50 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f486b600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4864119e10 con 0x7f486410fa50 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f4863e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f486410a410 0x7f48641a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f4863e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f486410a410 0x7f48641a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42604/0 (socket says 172.21.15.106:42604) 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f4868c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a50 0x7f48641a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f4863e00700 1 -- 172.21.15.106:0/659409917 learned_addr learned my addr 172.21.15.106:0/659409917 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.567+0000 7f4869600700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f486410fa50 0x7f48641a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.568+0000 7f4868c00700 1 -- 172.21.15.106:0/659409917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f486410fa50 msgr2=0x7f48641a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.568+0000 7f4868c00700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f486410fa50 0x7f48641a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.568+0000 7f4868c00700 1 -- 172.21.15.106:0/659409917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f486410a410 msgr2=0x7f48641a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.568+0000 7f4868c00700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f486410a410 0x7f48641a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.568+0000 7f4868c00700 1 -- 172.21.15.106:0/659409917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48641a50a0 con 0x7f4864109a50 2024-04-28T12:49:39.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.568+0000 7f4868c00700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a50 0x7f48641a00e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f485800efc0 tx=0x7f485800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:39.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.568+0000 7f4861600700 1 -- 172.21.15.106:0/659409917 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48580092e0 con 0x7f4864109a50 2024-04-28T12:49:39.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.568+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48641a5330 con 0x7f4864109a50 2024-04-28T12:49:39.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.569+0000 7f4861600700 1 -- 172.21.15.106:0/659409917 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4858003a00 con 0x7f4864109a50 2024-04-28T12:49:39.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.569+0000 7f4861600700 1 -- 172.21.15.106:0/659409917 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4858004630 con 0x7f4864109a50 2024-04-28T12:49:39.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.569+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48641acc10 con 0x7f4864109a50 2024-04-28T12:49:39.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.570+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486406a110 con 0x7f4864109a50 2024-04-28T12:49:39.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.572+0000 7f4861600700 1 -- 172.21.15.106:0/659409917 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f485801b030 con 0x7f4864109a50 2024-04-28T12:49:39.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.573+0000 7f4861600700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f484c0a3cd0 0x7f484c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:39.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.573+0000 7f4861600700 1 -- 172.21.15.106:0/659409917 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f48580d0240 con 0x7f4864109a50 2024-04-28T12:49:39.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.573+0000 7f4863e00700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f484c0a3cd0 0x7f484c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:39.570 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.574+0000 7f4863e00700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f484c0a3cd0 0x7f484c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4864078ad0 tx=0x7f4850009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:39.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.578+0000 7f4861600700 1 -- 172.21.15.106:0/659409917 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4858022020 con 0x7f4864109a50 2024-04-28T12:49:39.867 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.870+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f486402e490 con 0x7f4864109a50 2024-04-28T12:49:39.873 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.877+0000 7f4861600700 1 -- 172.21.15.106:0/659409917 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+265007 (secure 0 0 0) 0x7f4858010040 con 0x7f4864109a50 2024-04-28T12:49:39.878 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f484c0a3cd0 msgr2=0x7f484c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f484c0a3cd0 0x7f484c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4864078ad0 tx=0x7f4850009260 comp rx=0 tx=0).stop 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a50 msgr2=0x7f48641a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a50 0x7f48641a00e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f485800efc0 tx=0x7f485800c3a0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 shutdown_connections 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f484c0a3cd0 0x7f484c0a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f486410fa50 0x7f48641a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4864109a50 0x7f48641a00e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 --2- 172.21.15.106:0/659409917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f486410a410 0x7f48641a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 >> 172.21.15.106:0/659409917 conn(0x7f4864076bf0 msgr2=0x7f48640771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 shutdown_connections 2024-04-28T12:49:39.880 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:39.883+0000 7f486b600700 1 -- 172.21.15.106:0/659409917 wait complete. 2024-04-28T12:49:40.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:40 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/659409917' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-28T12:49:40.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:40 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/659409917' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-28T12:49:40.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:40 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/659409917' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-28T12:49:40.717 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1556498064},{"type":"v1","addr":"172.21.15.106:6801","nonce":1556498064}]},"active_addr":"172.21.15.106:6801/1556498064","active_change":"2024-04-28T12:40:55.904288+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.106:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":4262362809}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":3530581705}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":1844533001}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":3628148950}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":1489035663}]}]} 2024-04-28T12:49:40.722 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-28T12:49:40.722 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-28T12:49:40.722 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-28T12:49:41.213 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:41.383 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:41 smithi106 ceph-mon[28081]: pgmap v341: 129 pgs: 129 active+clean; 629 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-28T12:49:41.396 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:41 smithi171 ceph-mon[63127]: pgmap v341: 129 pgs: 129 active+clean; 629 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-28T12:49:41.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:41 smithi129 ceph-mon[62418]: pgmap v341: 129 pgs: 129 active+clean; 629 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-28T12:49:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.324+0000 7fcdf1200700 1 -- 172.21.15.106:0/368476637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdec06ba50 msgr2=0x7fcdec10f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.324+0000 7fcdf1200700 1 --2- 172.21.15.106:0/368476637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdec06ba50 0x7fcdec10f990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009990 tx=0x7fcdd4009ca0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.324+0000 7fcdf1200700 1 -- 172.21.15.106:0/368476637 shutdown_connections 2024-04-28T12:49:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.324+0000 7fcdf1200700 1 --2- 172.21.15.106:0/368476637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec10fed0 0x7fcdec1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.324+0000 7fcdf1200700 1 --2- 172.21.15.106:0/368476637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdec06b130 0x7fcdec06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.324+0000 7fcdf1200700 1 --2- 172.21.15.106:0/368476637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdec06ba50 0x7fcdec10f990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.324+0000 7fcdf1200700 1 -- 172.21.15.106:0/368476637 >> 172.21.15.106:0/368476637 conn(0x7fcdec0792d0 msgr2=0x7fcdec0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.325+0000 7fcdf1200700 1 -- 172.21.15.106:0/368476637 shutdown_connections 2024-04-28T12:49:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.325+0000 7fcdf1200700 1 -- 172.21.15.106:0/368476637 wait complete. 2024-04-28T12:49:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.326+0000 7fcdf1200700 1 Processor -- start 2024-04-28T12:49:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.326+0000 7fcdf1200700 1 -- start start 2024-04-28T12:49:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.326+0000 7fcdf1200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdec06b130 0x7fcdec1a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.326+0000 7fcdf1200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdec06ba50 0x7fcdec1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.326+0000 7fcdf1200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec10fed0 0x7fcdec19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.326+0000 7fcdf1200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec115800 con 0x7fcdec10fed0 2024-04-28T12:49:43.324 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.326+0000 7fcdf1200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec115680 con 0x7fcdec06ba50 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.326+0000 7fcdf1200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec115980 con 0x7fcdec06b130 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.326+0000 7fcdea800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec10fed0 0x7fcdec19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcde9e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdec06b130 0x7fcdec1a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcdea800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec10fed0 0x7fcdec19bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42626/0 (socket says 172.21.15.106:42626) 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcdea800700 1 -- 172.21.15.106:0/3243809834 learned_addr learned my addr 172.21.15.106:0/3243809834 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcde1400700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdec06ba50 0x7fcdec1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcde9e00700 1 -- 172.21.15.106:0/3243809834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdec06ba50 msgr2=0x7fcdec1a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcde9e00700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdec06ba50 0x7fcdec1a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcde9e00700 1 -- 172.21.15.106:0/3243809834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec10fed0 msgr2=0x7fcdec19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcde9e00700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec10fed0 0x7fcdec19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcde9e00700 1 -- 172.21.15.106:0/3243809834 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec19c440 con 0x7fcdec06b130 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.327+0000 7fcde9e00700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdec06b130 0x7fcdec1a1a40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e9f0 tx=0x7fcddc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:43.325 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.328+0000 7fcde3400700 1 -- 172.21.15.106:0/3243809834 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc004220 con 0x7fcdec06b130 2024-04-28T12:49:43.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.328+0000 7fcde3400700 1 -- 172.21.15.106:0/3243809834 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcddc0043c0 con 0x7fcdec06b130 2024-04-28T12:49:43.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.328+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec19c730 con 0x7fcdec06b130 2024-04-28T12:49:43.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.328+0000 7fcde3400700 1 -- 172.21.15.106:0/3243809834 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc005020 con 0x7fcdec06b130 2024-04-28T12:49:43.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.328+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1a89c0 con 0x7fcdec06b130 2024-04-28T12:49:43.326 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.329+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec102940 con 0x7fcdec06b130 2024-04-28T12:49:43.327 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.331+0000 7fcde3400700 1 -- 172.21.15.106:0/3243809834 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fcddc004750 con 0x7fcdec06b130 2024-04-28T12:49:43.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.332+0000 7fcde3400700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcdd00a3cd0 0x7fcdd00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:43.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.332+0000 7fcde3400700 1 -- 172.21.15.106:0/3243809834 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fcddc0d07b0 con 0x7fcdec06b130 2024-04-28T12:49:43.328 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.332+0000 7fcde1400700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcdd00a3cd0 0x7fcdd00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:43.329 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.333+0000 7fcde1400700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcdd00a3cd0 0x7fcdd00a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40098e0 tx=0x7fcdd4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:43.332 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.337+0000 7fcde3400700 1 -- 172.21.15.106:0/3243809834 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcddc09cd90 con 0x7fcdec06b130 2024-04-28T12:49:43.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:43 smithi171 ceph-mon[63127]: pgmap v342: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 22 op/s 2024-04-28T12:49:43.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:43 smithi129 ceph-mon[62418]: pgmap v342: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 22 op/s 2024-04-28T12:49:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.563+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdec19c960 con 0x7fcdec06b130 2024-04-28T12:49:43.560 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:43 smithi106 ceph-mon[28081]: pgmap v342: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 22 op/s 2024-04-28T12:49:43.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.565+0000 7fcde3400700 1 -- 172.21.15.106:0/3243809834 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7fcddc09c690 con 0x7fcdec06b130 2024-04-28T12:49:43.563 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:43.563 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.568+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcdd00a3cd0 msgr2=0x7fcdd00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.568+0000 7fcdf1200700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcdd00a3cd0 0x7fcdd00a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40098e0 tx=0x7fcdd4005960 comp rx=0 tx=0).stop 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdec06b130 msgr2=0x7fcdec1a1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdec06b130 0x7fcdec1a1a40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e9f0 tx=0x7fcddc00ed00 comp rx=0 tx=0).stop 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 shutdown_connections 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdec10fed0 0x7fcdec19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcdd00a3cd0 0x7fcdd00a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdec06b130 0x7fcdec1a1a40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 --2- 172.21.15.106:0/3243809834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcdec06ba50 0x7fcdec1a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 >> 172.21.15.106:0/3243809834 conn(0x7fcdec0792d0 msgr2=0x7fcdec1113c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 shutdown_connections 2024-04-28T12:49:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:43.569+0000 7fcdf1200700 1 -- 172.21.15.106:0/3243809834 wait complete. 2024-04-28T12:49:44.312 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:44 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3243809834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:44.388 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-28T12:49:44.388 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-28T12:49:44.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:44 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3243809834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:44.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:44 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3243809834' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:44.827 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:45.303 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:45 smithi106 ceph-mon[28081]: pgmap v343: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-04-28T12:49:45.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:45 smithi171 ceph-mon[63127]: pgmap v343: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-04-28T12:49:45.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:45 smithi129 ceph-mon[62418]: pgmap v343: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-04-28T12:49:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.894+0000 7f33c6600700 1 -- 172.21.15.106:0/1154969708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33c0075f70 msgr2=0x7f33c01140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.894+0000 7f33c6600700 1 --2- 172.21.15.106:0/1154969708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33c0075f70 0x7f33c01140b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009990 tx=0x7f33a8009ca0 comp rx=0 tx=0).stop 2024-04-28T12:49:46.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.894+0000 7f33c6600700 1 -- 172.21.15.106:0/1154969708 shutdown_connections 2024-04-28T12:49:46.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.894+0000 7f33c6600700 1 --2- 172.21.15.106:0/1154969708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c01145f0 0x7f33c01169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:46.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.894+0000 7f33c6600700 1 --2- 172.21.15.106:0/1154969708 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33c0075650 0x7f33c0075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:46.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.894+0000 7f33c6600700 1 --2- 172.21.15.106:0/1154969708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33c0075f70 0x7f33c01140b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:46.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.894+0000 7f33c6600700 1 -- 172.21.15.106:0/1154969708 >> 172.21.15.106:0/1154969708 conn(0x7f33c007a580 msgr2=0x7f33c007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:46.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.894+0000 7f33c6600700 1 -- 172.21.15.106:0/1154969708 shutdown_connections 2024-04-28T12:49:46.892 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.894+0000 7f33c6600700 1 -- 172.21.15.106:0/1154969708 wait complete. 2024-04-28T12:49:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.895+0000 7f33c6600700 1 Processor -- start 2024-04-28T12:49:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.896+0000 7f33c6600700 1 -- start start 2024-04-28T12:49:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.896+0000 7f33c6600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33c0075650 0x7f33c01a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.896+0000 7f33c6600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c0075f70 0x7f33c01a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.896+0000 7f33c6600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33c01145f0 0x7f33c01a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.896+0000 7f33c6600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0119c00 con 0x7f33c0075f70 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.896+0000 7f33c6600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0119a80 con 0x7f33c01145f0 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.896+0000 7f33c6600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0119d80 con 0x7f33c0075650 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33b6a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c0075f70 0x7f33c01a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33bf400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33c0075650 0x7f33c01a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33bf400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33c0075650 0x7f33c01a01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:35736/0 (socket says 172.21.15.106:35736) 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33bf400700 1 -- 172.21.15.106:0/58379137 learned_addr learned my addr 172.21.15.106:0/58379137 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33b6a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c0075f70 0x7f33c01a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42640/0 (socket says 172.21.15.106:42640) 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33bfe00700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33c01145f0 0x7f33c01a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33bf400700 1 -- 172.21.15.106:0/58379137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33c01145f0 msgr2=0x7f33c01a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33bf400700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33c01145f0 0x7f33c01a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33bf400700 1 -- 172.21.15.106:0/58379137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c0075f70 msgr2=0x7f33c01a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33bf400700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c0075f70 0x7f33c01a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.897+0000 7f33bf400700 1 -- 172.21.15.106:0/58379137 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c01a51c0 con 0x7f33c0075650 2024-04-28T12:49:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.898+0000 7f33bf400700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33c0075650 0x7f33c01a01b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e9c0 tx=0x7f33b000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.898+0000 7f33bcc00700 1 -- 172.21.15.106:0/58379137 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b000cbd0 con 0x7f33c0075650 2024-04-28T12:49:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.898+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c01a54b0 con 0x7f33c0075650 2024-04-28T12:49:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.898+0000 7f33bcc00700 1 -- 172.21.15.106:0/58379137 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33b000ee50 con 0x7f33c0075650 2024-04-28T12:49:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.898+0000 7f33bcc00700 1 -- 172.21.15.106:0/58379137 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0005550 con 0x7f33c0075650 2024-04-28T12:49:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.898+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c01acd30 con 0x7f33c0075650 2024-04-28T12:49:46.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.899+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c006a110 con 0x7f33c0075650 2024-04-28T12:49:46.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.901+0000 7f33bcc00700 1 -- 172.21.15.106:0/58379137 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f33b0004750 con 0x7f33c0075650 2024-04-28T12:49:46.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.902+0000 7f33bcc00700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f33a40a3cd0 0x7f33a40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:46.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.903+0000 7f33bcc00700 1 -- 172.21.15.106:0/58379137 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f33b00d0ac0 con 0x7f33c0075650 2024-04-28T12:49:46.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.903+0000 7f33b6a00700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f33a40a3cd0 0x7f33a40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:46.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.903+0000 7f33b6a00700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f33a40a3cd0 0x7f33a40a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f33a80098e0 tx=0x7f33a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:46.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:46.907+0000 7f33bcc00700 1 -- 172.21.15.106:0/58379137 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f33b009d0a0 con 0x7f33c0075650 2024-04-28T12:49:47.136 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.140+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33c01a1510 con 0x7f33c0075650 2024-04-28T12:49:47.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.142+0000 7f33bcc00700 1 -- 172.21.15.106:0/58379137 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24197 (secure 0 0 0) 0x7f33b009c9a0 con 0x7f33c0075650 2024-04-28T12:49:47.139 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:49:47.139 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:48:59.630472+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:49:47.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f33a40a3cd0 msgr2=0x7f33a40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:47.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f33a40a3cd0 0x7f33a40a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f33a80098e0 tx=0x7f33a8005960 comp rx=0 tx=0).stop 2024-04-28T12:49:47.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33c0075650 msgr2=0x7f33c01a01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:47.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33c0075650 0x7f33c01a01b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e9c0 tx=0x7f33b000ecd0 comp rx=0 tx=0).stop 2024-04-28T12:49:47.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 shutdown_connections 2024-04-28T12:49:47.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33c01145f0 0x7f33c01a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:47.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f33a40a3cd0 0x7f33a40a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:47.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33c0075650 0x7f33c01a01b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:47.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 --2- 172.21.15.106:0/58379137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33c0075f70 0x7f33c01a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:47.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.145+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 >> 172.21.15.106:0/58379137 conn(0x7f33c007a580 msgr2=0x7f33c0113500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:47.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.146+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 shutdown_connections 2024-04-28T12:49:47.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:47.146+0000 7f33c6600700 1 -- 172.21.15.106:0/58379137 wait complete. 2024-04-28T12:49:47.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:47 smithi171 ceph-mon[63127]: pgmap v344: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-28T12:49:47.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:47 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/58379137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:47.510 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:47 smithi106 ceph-mon[28081]: pgmap v344: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-28T12:49:47.511 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:47 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/58379137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:47 smithi129 ceph-mon[62418]: pgmap v344: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-28T12:49:47.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:47 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/58379137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:49:47.886 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-28T12:49:47.886 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-28T12:49:47.887 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-28T12:49:47.887 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-28T12:49:47.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-28T12:49:47.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-28T12:49:47.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-28T12:49:47.889 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-28T12:49:47.889 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-28T12:49:47.890 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-28T12:49:47.890 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-28T12:49:47.891 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-28T12:49:48.913 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:48.979 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:48.981 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:48.982 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:48.993 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:49.002 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:49.035 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:49.072 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:49.085 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:49.138 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:49.141 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:49:49.468 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:49 smithi106 ceph-mon[28081]: pgmap v345: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-28T12:49:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:49 smithi171 ceph-mon[63127]: pgmap v345: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-28T12:49:49.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:49 smithi129 ceph-mon[62418]: pgmap v345: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-28T12:49:51.419 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:51 smithi106 ceph-mon[28081]: pgmap v346: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-28T12:49:51.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:51 smithi171 ceph-mon[63127]: pgmap v346: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-28T12:49:51.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:51 smithi129 ceph-mon[62418]: pgmap v346: 129 pgs: 129 active+clean; 633 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-28T12:49:53.395 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:53 smithi106 ceph-mon[28081]: pgmap v347: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-28T12:49:53.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:53 smithi171 ceph-mon[63127]: pgmap v347: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-28T12:49:53.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:53 smithi129 ceph-mon[62418]: pgmap v347: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-28T12:49:55.188 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:55 smithi106 ceph-mon[28081]: pgmap v348: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-28T12:49:55.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:55 smithi171 ceph-mon[63127]: pgmap v348: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-28T12:49:55.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:55 smithi129 ceph-mon[62418]: pgmap v348: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-28T12:49:56.346 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:49:56.347 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-28T12:49:56.347 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-04-28T12:49:56.347 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-28T12:49:56.347 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2024-04-28T12:49:56.347 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-04-28T12:49:56.347 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-28T12:49:56.347 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-28T12:49:56.347 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-04-28T12:49:56.347 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-28T12:49:56.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:49:56.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-28T12:49:56.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-04-28T12:49:56.420 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-28T12:49:56.421 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2024-04-28T12:49:56.421 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-04-28T12:49:56.421 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-28T12:49:56.421 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-28T12:49:56.421 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-04-28T12:49:56.421 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-28T12:49:56.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:49:56.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-28T12:49:56.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]: dispatch 2024-04-28T12:49:56.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-28T12:49:56.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]: dispatch 2024-04-28T12:49:56.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-04-28T12:49:56.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-28T12:49:56.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-28T12:49:56.429 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]: dispatch 2024-04-28T12:49:56.430 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-28T12:49:57.458 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: pgmap v349: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-28T12:49:57.458 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-28T12:49:57.458 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-04-28T12:49:57.458 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: osdmap e90: 12 total, 12 up, 12 in 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.459 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: pgmap v349: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-28T12:49:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-28T12:49:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-04-28T12:49:57.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: osdmap e90: 12 total, 12 up, 12 in 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: pgmap v349: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-28T12:49:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-28T12:49:57.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4, 8, 11]}]': finished 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 8]}]': finished 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 6]}]': finished 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: osdmap e90: 12 total, 12 up, 12 in 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:57.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:49:58.390 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.393+0000 7f4bcf400700 1 -- 172.21.15.106:0/138981537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bc8075f70 msgr2=0x7f4bc810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.393+0000 7f4bcf400700 1 --2- 172.21.15.106:0/138981537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bc8075f70 0x7f4bc810f8e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009990 tx=0x7f4bb4009ca0 comp rx=0 tx=0).stop 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.393+0000 7f4bcf400700 1 -- 172.21.15.106:0/138981537 shutdown_connections 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.393+0000 7f4bcf400700 1 --2- 172.21.15.106:0/138981537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bc810fe20 0x7f4bc8112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.393+0000 7f4bcf400700 1 --2- 172.21.15.106:0/138981537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc8075650 0x7f4bc8075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.393+0000 7f4bcf400700 1 --2- 172.21.15.106:0/138981537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bc8075f70 0x7f4bc810f8e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.393+0000 7f4bcf400700 1 -- 172.21.15.106:0/138981537 >> 172.21.15.106:0/138981537 conn(0x7f4bc807a580 msgr2=0x7f4bc807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.393+0000 7f4bcf400700 1 -- 172.21.15.106:0/138981537 shutdown_connections 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.393+0000 7f4bcf400700 1 -- 172.21.15.106:0/138981537 wait complete. 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.394+0000 7f4bcf400700 1 Processor -- start 2024-04-28T12:49:58.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.394+0000 7f4bcf400700 1 -- start start 2024-04-28T12:49:58.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcf400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc8075650 0x7f4bc81a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:58.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcf400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bc8075f70 0x7f4bc81a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:58.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcf400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bc810fe20 0x7f4bc819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:58.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcf400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc81157e0 con 0x7f4bc8075650 2024-04-28T12:49:58.393 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcf400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc8115660 con 0x7f4bc8075f70 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcf400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc8115960 con 0x7f4bc810fe20 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcd400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bc810fe20 0x7f4bc819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcca00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc8075650 0x7f4bc81a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bc7e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bc8075f70 0x7f4bc81a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcd400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bc810fe20 0x7f4bc819bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:45448/0 (socket says 172.21.15.106:45448) 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.395+0000 7f4bcd400700 1 -- 172.21.15.106:0/2622457101 learned_addr learned my addr 172.21.15.106:0/2622457101 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bcd400700 1 -- 172.21.15.106:0/2622457101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bc8075f70 msgr2=0x7f4bc81a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bcd400700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bc8075f70 0x7f4bc81a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bcd400700 1 -- 172.21.15.106:0/2622457101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc8075650 msgr2=0x7f4bc81a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bcd400700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bc8075650 0x7f4bc81a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bcd400700 1 -- 172.21.15.106:0/2622457101 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc819c400 con 0x7f4bc810fe20 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bc7e00700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bc8075f70 0x7f4bc81a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:49:58.394 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bcd400700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bc810fe20 0x7f4bc819bb20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c110 tx=0x7f4bc000c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:58.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bc5600700 1 -- 172.21.15.106:0/2622457101 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00092e0 con 0x7f4bc810fe20 2024-04-28T12:49:58.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bc5600700 1 -- 172.21.15.106:0/2622457101 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bc0003a00 con 0x7f4bc810fe20 2024-04-28T12:49:58.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc819c690 con 0x7f4bc810fe20 2024-04-28T12:49:58.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bc5600700 1 -- 172.21.15.106:0/2622457101 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0004650 con 0x7f4bc810fe20 2024-04-28T12:49:58.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.396+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc81a89d0 con 0x7f4bc810fe20 2024-04-28T12:49:58.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.397+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bc819c9d0 con 0x7f4bc810fe20 2024-04-28T12:49:58.395 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.399+0000 7f4bc5600700 1 -- 172.21.15.106:0/2622457101 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4bc001a020 con 0x7f4bc810fe20 2024-04-28T12:49:58.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.400+0000 7f4bc5600700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4bb00a3d50 0x7f4bb00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:58.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.400+0000 7f4bc5600700 1 -- 172.21.15.106:0/2622457101 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f4bc00d0140 con 0x7f4bc810fe20 2024-04-28T12:49:58.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.401+0000 7f4bcca00700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4bb00a3d50 0x7f4bb00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:58.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.401+0000 7f4bc5600700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] conn(0x7f4bb00aee30 0x7f4bb00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:49:58.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.401+0000 7f4bc5600700 1 -- 172.21.15.106:0/2622457101 --> [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4bc0009c20 con 0x7f4bb00aee30 2024-04-28T12:49:58.397 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.401+0000 7f4bc5600700 1 -- 172.21.15.106:0/2622457101 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4bc00d7050 con 0x7f4bc810fe20 2024-04-28T12:49:58.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.401+0000 7f4bc7e00700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] conn(0x7f4bb00aee30 0x7f4bb00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:49:58.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.401+0000 7f4bcca00700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4bb00a3d50 0x7f4bb00a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009c60 tx=0x7f4bbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:58.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.402+0000 7f4bc7e00700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] conn(0x7f4bb00aee30 0x7f4bb00b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:49:58.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.403+0000 7f4bc5600700 1 -- 172.21.15.106:0/2622457101 <== osd.11 v2:172.21.15.171:6824/2516393366 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4bc0009c20 con 0x7f4bb00aee30 2024-04-28T12:49:58.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.420+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 --> [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bc8063e70 con 0x7f4bb00aee30 2024-04-28T12:49:58.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.421+0000 7f4bc5600700 1 -- 172.21.15.106:0/2622457101 <== osd.11 v2:172.21.15.171:6824/2516393366 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4bc8063e70 con 0x7f4bb00aee30 2024-04-28T12:49:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.421+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 >> [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] conn(0x7f4bb00aee30 msgr2=0x7f4bb00b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.421+0000 7f4bcf400700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] conn(0x7f4bb00aee30 0x7f4bb00b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:49:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.422+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4bb00a3d50 msgr2=0x7f4bb00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.422+0000 7f4bcf400700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4bb00a3d50 0x7f4bb00a6210 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009c60 tx=0x7f4bbc009260 comp rx=0 tx=0).stop 2024-04-28T12:49:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.422+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bc810fe20 msgr2=0x7f4bc819bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:49:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.422+0000 7f4bcf400700 1 --2- 172.21.15.106:0/2622457101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4bc810fe20 0x7f4bc819bb20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000c110 tx=0x7f4bc000c420 comp rx=0 tx=0).stop 2024-04-28T12:49:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.422+0000 7f4bcd400700 1 -- 172.21.15.106:0/2622457101 reap_dead start 2024-04-28T12:49:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.422+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 shutdown_connections 2024-04-28T12:49:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.422+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 >> 172.21.15.106:0/2622457101 conn(0x7f4bc807a580 msgr2=0x7f4bc8100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:49:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.422+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 shutdown_connections 2024-04-28T12:49:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:49:58.422+0000 7f4bcf400700 1 -- 172.21.15.106:0/2622457101 wait complete. 2024-04-28T12:49:58.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:58 smithi171 ceph-mon[63127]: osdmap e91: 12 total, 12 up, 12 in 2024-04-28T12:49:58.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:58 smithi129 ceph-mon[62418]: osdmap e91: 12 total, 12 up, 12 in 2024-04-28T12:49:58.540 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:58 smithi106 ceph-mon[28081]: osdmap e91: 12 total, 12 up, 12 in 2024-04-28T12:49:59.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:49:59 smithi171 ceph-mon[63127]: pgmap v352: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 10 op/s 2024-04-28T12:49:59.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:49:59 smithi129 ceph-mon[62418]: pgmap v352: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 10 op/s 2024-04-28T12:49:59.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:49:59 smithi106 ceph-mon[28081]: pgmap v352: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 383 B/s wr, 10 op/s 2024-04-28T12:50:00.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:00 smithi106 ceph-mon[28081]: overall HEALTH_OK 2024-04-28T12:50:00.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:00 smithi171 ceph-mon[63127]: overall HEALTH_OK 2024-04-28T12:50:00.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:00 smithi129 ceph-mon[62418]: overall HEALTH_OK 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.870+0000 7f4a67200700 1 -- 172.21.15.106:0/678323018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a68104570 msgr2=0x7f4a68104950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.870+0000 7f4a67200700 1 --2- 172.21.15.106:0/678323018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a68104570 0x7f4a68104950 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009990 tx=0x7f4a58009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.870+0000 7f4a67200700 1 -- 172.21.15.106:0/678323018 shutdown_connections 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.870+0000 7f4a67200700 1 --2- 172.21.15.106:0/678323018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a6810d950 0x7f4a6810fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.870+0000 7f4a67200700 1 --2- 172.21.15.106:0/678323018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a68104570 0x7f4a68104950 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.870+0000 7f4a67200700 1 --2- 172.21.15.106:0/678323018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a68104e90 0x7f4a6810d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.870+0000 7f4a67200700 1 -- 172.21.15.106:0/678323018 >> 172.21.15.106:0/678323018 conn(0x7f4a680fdd40 msgr2=0x7f4a68100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.870+0000 7f4a67200700 1 -- 172.21.15.106:0/678323018 shutdown_connections 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.870+0000 7f4a67200700 1 -- 172.21.15.106:0/678323018 wait complete. 2024-04-28T12:50:00.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.871+0000 7f4a67200700 1 Processor -- start 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.871+0000 7f4a67200700 1 -- start start 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.871+0000 7f4a67200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a68104570 0x7f4a6819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a67200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a68104e90 0x7f4a6819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a67200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a6810d950 0x7f4a681998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a67200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4a68113570 con 0x7f4a68104e90 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a67200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681133f0 con 0x7f4a6810d950 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a67200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4a681136f0 con 0x7f4a68104570 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a68104e90 0x7f4a6819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a68104570 0x7f4a6819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a68104e90 0x7f4a6819fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41622/0 (socket says 172.21.15.106:41622) 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a68104570 0x7f4a6819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:41722/0 (socket says 172.21.15.106:41722) 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65400700 1 -- 172.21.15.106:0/4272897218 learned_addr learned my addr 172.21.15.106:0/4272897218 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a66800700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a6810d950 0x7f4a681998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65e00700 1 -- 172.21.15.106:0/4272897218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a6810d950 msgr2=0x7f4a681998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65e00700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a6810d950 0x7f4a681998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65e00700 1 -- 172.21.15.106:0/4272897218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a68104e90 msgr2=0x7f4a6819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65e00700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a68104e90 0x7f4a6819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65e00700 1 -- 172.21.15.106:0/4272897218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6819a010 con 0x7f4a68104570 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.872+0000 7f4a65400700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a68104e90 0x7f4a6819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.873+0000 7f4a65e00700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a68104570 0x7f4a6819f6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4a58000c00 tx=0x7f4a580047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.873+0000 7f4a5ea00700 1 -- 172.21.15.106:0/4272897218 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58026070 con 0x7f4a68104570 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.873+0000 7f4a5ea00700 1 -- 172.21.15.106:0/4272897218 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a58021410 con 0x7f4a68104570 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.873+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6819a2a0 con 0x7f4a68104570 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.873+0000 7f4a5ea00700 1 -- 172.21.15.106:0/4272897218 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a58017650 con 0x7f4a68104570 2024-04-28T12:50:00.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.873+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a681a65c0 con 0x7f4a68104570 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.874+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a68063bb0 con 0x7f4a68104570 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.876+0000 7f4a5ea00700 1 -- 172.21.15.106:0/4272897218 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4a58029070 con 0x7f4a68104570 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.878+0000 7f4a5ea00700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4a4c0a3cd0 0x7f4a4c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.878+0000 7f4a5ea00700 1 -- 172.21.15.106:0/4272897218 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f4a580d1fe0 con 0x7f4a68104570 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.878+0000 7f4a65400700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4a4c0a3cd0 0x7f4a4c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.878+0000 7f4a5ea00700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] conn(0x7f4a4c0aedb0 0x7f4a4c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.878+0000 7f4a5ea00700 1 -- 172.21.15.106:0/4272897218 --> [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a4c0b1920 con 0x7f4a4c0aedb0 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.878+0000 7f4a5ea00700 1 -- 172.21.15.106:0/4272897218 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4a580d2390 con 0x7f4a68104570 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.879+0000 7f4a65400700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4a4c0a3cd0 0x7f4a4c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a6810ec80 tx=0x7f4a54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.879+0000 7f4a66800700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] conn(0x7f4a4c0aedb0 0x7f4a4c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.879+0000 7f4a66800700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] conn(0x7f4a4c0aedb0 0x7f4a4c0b1250 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.880+0000 7f4a5ea00700 1 -- 172.21.15.106:0/4272897218 <== osd.10 v2:172.21.15.129:6824/3282354257 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a4c0b1920 con 0x7f4a4c0aedb0 2024-04-28T12:50:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.898+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 --> [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a68064030 con 0x7f4a4c0aedb0 2024-04-28T12:50:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.899+0000 7f4a5ea00700 1 -- 172.21.15.106:0/4272897218 <== osd.10 v2:172.21.15.129:6824/3282354257 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a68064030 con 0x7f4a4c0aedb0 2024-04-28T12:50:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.899+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 >> [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] conn(0x7f4a4c0aedb0 msgr2=0x7f4a4c0b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.899+0000 7f4a67200700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] conn(0x7f4a4c0aedb0 0x7f4a4c0b1250 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.900+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4a4c0a3cd0 msgr2=0x7f4a4c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.900+0000 7f4a67200700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4a4c0a3cd0 0x7f4a4c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a6810ec80 tx=0x7f4a54009260 comp rx=0 tx=0).stop 2024-04-28T12:50:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.900+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a68104570 msgr2=0x7f4a6819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.900+0000 7f4a67200700 1 --2- 172.21.15.106:0/4272897218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a68104570 0x7f4a6819f6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4a58000c00 tx=0x7f4a580047b0 comp rx=0 tx=0).stop 2024-04-28T12:50:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.900+0000 7f4a66800700 1 -- 172.21.15.106:0/4272897218 reap_dead start 2024-04-28T12:50:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.900+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 shutdown_connections 2024-04-28T12:50:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.900+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 >> 172.21.15.106:0/4272897218 conn(0x7f4a680fdd40 msgr2=0x7f4a680feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.900+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 shutdown_connections 2024-04-28T12:50:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.900+0000 7f4a67200700 1 -- 172.21.15.106:0/4272897218 wait complete. 2024-04-28T12:50:01.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.184+0000 7fe53c800700 1 -- 172.21.15.106:0/146591471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5341094c0 msgr2=0x7fe5341098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:01.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.184+0000 7fe53c800700 1 --2- 172.21.15.106:0/146591471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5341094c0 0x7fe5341098a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00b0a0 tx=0x7fe52c00b3b0 comp rx=0 tx=0).stop 2024-04-28T12:50:01.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.185+0000 7fe53c800700 1 -- 172.21.15.106:0/146591471 shutdown_connections 2024-04-28T12:50:01.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.185+0000 7fe53c800700 1 --2- 172.21.15.106:0/146591471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe534103e30 0x7fe534110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:01.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.185+0000 7fe53c800700 1 --2- 172.21.15.106:0/146591471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5341094c0 0x7fe5341098a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:01.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.185+0000 7fe53c800700 1 --2- 172.21.15.106:0/146591471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe534103470 0x7fe5341038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:01.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.185+0000 7fe53c800700 1 -- 172.21.15.106:0/146591471 >> 172.21.15.106:0/146591471 conn(0x7fe53407a580 msgr2=0x7fe53407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.185+0000 7fe53c800700 1 -- 172.21.15.106:0/146591471 shutdown_connections 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.185+0000 7fe53c800700 1 -- 172.21.15.106:0/146591471 wait complete. 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.186+0000 7fe53c800700 1 Processor -- start 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.186+0000 7fe53c800700 1 -- start start 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.186+0000 7fe53c800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534103470 0x7fe53419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.186+0000 7fe53c800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe534103e30 0x7fe53419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.186+0000 7fe53c800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5341094c0 0x7fe5341a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.186+0000 7fe53c800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe534113610 con 0x7fe534103470 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.186+0000 7fe53c800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe534113490 con 0x7fe534103e30 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.186+0000 7fe53c800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe534113790 con 0x7fe5341094c0 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534103470 0x7fe53419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534103470 0x7fe53419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41612/0 (socket says 172.21.15.106:41612) 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539e00700 1 -- 172.21.15.106:0/1976530098 learned_addr learned my addr 172.21.15.106:0/1976530098 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539400700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe534103e30 0x7fe53419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe53a800700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5341094c0 0x7fe5341a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539e00700 1 -- 172.21.15.106:0/1976530098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5341094c0 msgr2=0x7fe5341a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539e00700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5341094c0 0x7fe5341a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539e00700 1 -- 172.21.15.106:0/1976530098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe534103e30 msgr2=0x7fe53419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539e00700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe534103e30 0x7fe53419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539e00700 1 -- 172.21.15.106:0/1976530098 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5341a0d90 con 0x7fe534103470 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe53a800700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5341094c0 0x7fe5341a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.187+0000 7fe539e00700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534103470 0x7fe53419bd80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00d010 tx=0x7fe52c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.188+0000 7fe52aa00700 1 -- 172.21.15.106:0/1976530098 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c021070 con 0x7fe534103470 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.188+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5341a1020 con 0x7fe534103470 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.188+0000 7fe52aa00700 1 -- 172.21.15.106:0/1976530098 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe52c007c40 con 0x7fe534103470 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.188+0000 7fe52aa00700 1 -- 172.21.15.106:0/1976530098 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c004710 con 0x7fe534103470 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.188+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5341a8900 con 0x7fe534103470 2024-04-28T12:50:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.189+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe534063bb0 con 0x7fe534103470 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.191+0000 7fe52aa00700 1 -- 172.21.15.106:0/1976530098 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe52c003a00 con 0x7fe534103470 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.192+0000 7fe52aa00700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe5200a3cd0 0x7fe5200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.192+0000 7fe52aa00700 1 -- 172.21.15.106:0/1976530098 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fe52c00f020 con 0x7fe534103470 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.193+0000 7fe539400700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe5200a3cd0 0x7fe5200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.193+0000 7fe52aa00700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] conn(0x7fe5200aedb0 0x7fe5200b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.193+0000 7fe52aa00700 1 -- 172.21.15.106:0/1976530098 --> [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe52c01fe80 con 0x7fe5200aedb0 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.193+0000 7fe52aa00700 1 -- 172.21.15.106:0/1976530098 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe52c0d2410 con 0x7fe534103470 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.193+0000 7fe53a800700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] conn(0x7fe5200aedb0 0x7fe5200b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.193+0000 7fe539400700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe5200a3cd0 0x7fe5200a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe524009680 tx=0x7fe524009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.194+0000 7fe53a800700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] conn(0x7fe5200aedb0 0x7fe5200b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.195+0000 7fe52aa00700 1 -- 172.21.15.106:0/1976530098 <== osd.1 v2:172.21.15.129:6800/3542000402 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe52c01fe80 con 0x7fe5200aedb0 2024-04-28T12:50:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.212+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 --> [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe534063fe0 con 0x7fe5200aedb0 2024-04-28T12:50:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.213+0000 7fe52aa00700 1 -- 172.21.15.106:0/1976530098 <== osd.1 v2:172.21.15.129:6800/3542000402 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe534063fe0 con 0x7fe5200aedb0 2024-04-28T12:50:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.214+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 >> [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] conn(0x7fe5200aedb0 msgr2=0x7fe5200b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.214+0000 7fe53c800700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] conn(0x7fe5200aedb0 0x7fe5200b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.214+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe5200a3cd0 msgr2=0x7fe5200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.214+0000 7fe53c800700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe5200a3cd0 0x7fe5200a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe524009680 tx=0x7fe524009260 comp rx=0 tx=0).stop 2024-04-28T12:50:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.214+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534103470 msgr2=0x7fe53419bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.214+0000 7fe53c800700 1 --2- 172.21.15.106:0/1976530098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534103470 0x7fe53419bd80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00d010 tx=0x7fe52c004d00 comp rx=0 tx=0).stop 2024-04-28T12:50:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.214+0000 7fe53a800700 1 -- 172.21.15.106:0/1976530098 reap_dead start 2024-04-28T12:50:01.483 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:01 smithi106 ceph-mon[28081]: pgmap v353: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:01.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.214+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 shutdown_connections 2024-04-28T12:50:01.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.214+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 >> 172.21.15.106:0/1976530098 conn(0x7fe53407a580 msgr2=0x7fe53410e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:01.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.215+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 shutdown_connections 2024-04-28T12:50:01.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:00.215+0000 7fe53c800700 1 -- 172.21.15.106:0/1976530098 wait complete. 2024-04-28T12:50:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:01 smithi171 ceph-mon[63127]: pgmap v353: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:01.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:01 smithi129 ceph-mon[62418]: pgmap v353: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:02.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:02 smithi106 ceph-mon[28081]: pgmap v354: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:02.378 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:02.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:02 smithi171 ceph-mon[63127]: pgmap v354: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:02.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:02 smithi129 ceph-mon[62418]: pgmap v354: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.850+0000 7f814aa00700 1 -- 172.21.15.106:0/1381729127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f81441094c0 msgr2=0x7f81441098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.850+0000 7f814aa00700 1 --2- 172.21.15.106:0/1381729127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f81441094c0 0x7f81441098a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f81340098f0 tx=0x7f8134009c00 comp rx=0 tx=0).stop 2024-04-28T12:50:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.850+0000 7f814aa00700 1 -- 172.21.15.106:0/1381729127 shutdown_connections 2024-04-28T12:50:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.850+0000 7f814aa00700 1 --2- 172.21.15.106:0/1381729127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8144103e30 0x7f8144110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.850+0000 7f814aa00700 1 --2- 172.21.15.106:0/1381729127 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f81441094c0 0x7f81441098a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.850+0000 7f814aa00700 1 --2- 172.21.15.106:0/1381729127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8144103470 0x7f81441038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.850+0000 7f814aa00700 1 -- 172.21.15.106:0/1381729127 >> 172.21.15.106:0/1381729127 conn(0x7f814407a580 msgr2=0x7f814407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.850+0000 7f814aa00700 1 -- 172.21.15.106:0/1381729127 shutdown_connections 2024-04-28T12:50:04.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.850+0000 7f814aa00700 1 -- 172.21.15.106:0/1381729127 wait complete. 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.851+0000 7f814aa00700 1 Processor -- start 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.851+0000 7f814aa00700 1 -- start start 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f814aa00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8144103470 0x7f814419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f814aa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8144103e30 0x7f814419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f814aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81441094c0 0x7f81441999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f814aa00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8144113640 con 0x7f81441094c0 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f814aa00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f81441134c0 con 0x7f8144103470 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f814aa00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f81441137c0 con 0x7f8144103e30 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f8148a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81441094c0 0x7f81441999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f8148a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81441094c0 0x7f81441999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41632/0 (socket says 172.21.15.106:41632) 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f8148a00700 1 -- 172.21.15.106:0/795058148 learned_addr learned my addr 172.21.15.106:0/795058148 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f8143e00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8144103470 0x7f814419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.852+0000 7f8143400700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8144103e30 0x7f814419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:04.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f8148a00700 1 -- 172.21.15.106:0/795058148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8144103e30 msgr2=0x7f814419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f8148a00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8144103e30 0x7f814419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f8148a00700 1 -- 172.21.15.106:0/795058148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8144103470 msgr2=0x7f814419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f8148a00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8144103470 0x7f814419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f8148a00700 1 -- 172.21.15.106:0/795058148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814419a120 con 0x7f81441094c0 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f8148a00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81441094c0 0x7f81441999c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f813c00e3c0 tx=0x7f813c00e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f8140c00700 1 -- 172.21.15.106:0/795058148 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0090d0 con 0x7f81441094c0 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f8140c00700 1 -- 172.21.15.106:0/795058148 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f813c009270 con 0x7f81441094c0 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f8140c00700 1 -- 172.21.15.106:0/795058148 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c007df0 con 0x7f81441094c0 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814419a410 con 0x7f81441094c0 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.853+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81441a66d0 con 0x7f81441094c0 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.854+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8144063b30 con 0x7f81441094c0 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.857+0000 7f8140c00700 1 -- 172.21.15.106:0/795058148 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f813c012050 con 0x7f81441094c0 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.858+0000 7f8140c00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81280a3cd0 0x7f81280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:04.897 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.858+0000 7f8140c00700 1 -- 172.21.15.106:0/795058148 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f813c0d0560 con 0x7f81441094c0 2024-04-28T12:50:04.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.858+0000 7f8143e00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81280a3cd0 0x7f81280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:04.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.858+0000 7f8140c00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] conn(0x7f81280aedb0 0x7f81280b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:04.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.858+0000 7f8140c00700 1 -- 172.21.15.106:0/795058148 --> [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81280b1920 con 0x7f81280aedb0 2024-04-28T12:50:04.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.858+0000 7f8140c00700 1 -- 172.21.15.106:0/795058148 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f813c0d0910 con 0x7f81441094c0 2024-04-28T12:50:04.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.859+0000 7f8143400700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] conn(0x7f81280aedb0 0x7f81280b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:04.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.859+0000 7f8143e00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81280a3cd0 0x7f81280a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f814419ab00 tx=0x7f8134019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:04.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.859+0000 7f8143400700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] conn(0x7f81280aedb0 0x7f81280b1250 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:04.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.860+0000 7f8140c00700 1 -- 172.21.15.106:0/795058148 <== osd.9 v2:172.21.15.106:6826/95511724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81280b1920 con 0x7f81280aedb0 2024-04-28T12:50:04.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.878+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 --> [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8144063fb0 con 0x7f81280aedb0 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.879+0000 7f8140c00700 1 -- 172.21.15.106:0/795058148 <== osd.9 v2:172.21.15.106:6826/95511724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8144063fb0 con 0x7f81280aedb0 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.879+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] conn(0x7f81280aedb0 msgr2=0x7f81280b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.879+0000 7f814aa00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] conn(0x7f81280aedb0 0x7f81280b1250 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.879+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81280a3cd0 msgr2=0x7f81280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.879+0000 7f814aa00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81280a3cd0 0x7f81280a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f814419ab00 tx=0x7f8134019040 comp rx=0 tx=0).stop 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.879+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81441094c0 msgr2=0x7f81441999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.879+0000 7f814aa00700 1 --2- 172.21.15.106:0/795058148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f81441094c0 0x7f81441999c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f813c00e3c0 tx=0x7f813c00e6d0 comp rx=0 tx=0).stop 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.879+0000 7f8148a00700 1 -- 172.21.15.106:0/795058148 reap_dead start 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.880+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 shutdown_connections 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.880+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 >> 172.21.15.106:0/795058148 conn(0x7f814407a580 msgr2=0x7f8144102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:04.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.880+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 shutdown_connections 2024-04-28T12:50:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:04.880+0000 7f814aa00700 1 -- 172.21.15.106:0/795058148 wait complete. 2024-04-28T12:50:05.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.431+0000 7f819b000700 1 -- 172.21.15.106:0/2565251796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8194075f70 msgr2=0x7f819410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:05.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.431+0000 7f819b000700 1 --2- 172.21.15.106:0/2565251796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8194075f70 0x7f819410f8e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8180009990 tx=0x7f8180009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:05.428 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:05 smithi106 ceph-mon[28081]: pgmap v355: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.431+0000 7f819b000700 1 -- 172.21.15.106:0/2565251796 shutdown_connections 2024-04-28T12:50:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.431+0000 7f819b000700 1 --2- 172.21.15.106:0/2565251796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819410fe20 0x7f8194112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.431+0000 7f819b000700 1 --2- 172.21.15.106:0/2565251796 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8194075650 0x7f8194075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.431+0000 7f819b000700 1 --2- 172.21.15.106:0/2565251796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8194075f70 0x7f819410f8e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.431+0000 7f819b000700 1 -- 172.21.15.106:0/2565251796 >> 172.21.15.106:0/2565251796 conn(0x7f819407a5d0 msgr2=0x7f819407a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.431+0000 7f819b000700 1 -- 172.21.15.106:0/2565251796 shutdown_connections 2024-04-28T12:50:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.432+0000 7f819b000700 1 -- 172.21.15.106:0/2565251796 wait complete. 2024-04-28T12:50:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.433+0000 7f819b000700 1 Processor -- start 2024-04-28T12:50:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.433+0000 7f819b000700 1 -- start start 2024-04-28T12:50:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.433+0000 7f819b000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8194075650 0x7f819419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.433+0000 7f819b000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8194075f70 0x7f819419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.433+0000 7f819b000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819410fe20 0x7f81941a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.433+0000 7f819b000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f81941158b0 con 0x7f8194075f70 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.433+0000 7f819b000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8194115730 con 0x7f819410fe20 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.433+0000 7f819b000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8194115a30 con 0x7f8194075650 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8193e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8194075650 0x7f819419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8193400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8194075f70 0x7f819419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8193e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8194075650 0x7f819419be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:41760/0 (socket says 172.21.15.106:41760) 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8193e00700 1 -- 172.21.15.106:0/2827714140 learned_addr learned my addr 172.21.15.106:0/2827714140 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8199000700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819410fe20 0x7f81941a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8193e00700 1 -- 172.21.15.106:0/2827714140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819410fe20 msgr2=0x7f81941a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8193e00700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f819410fe20 0x7f81941a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8193e00700 1 -- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8194075f70 msgr2=0x7f819419c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8193e00700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8194075f70 0x7f819419c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.434+0000 7f8193e00700 1 -- 172.21.15.106:0/2827714140 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81941a0e30 con 0x7f8194075650 2024-04-28T12:50:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.435+0000 7f8193e00700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8194075650 0x7f819419be20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f818400e9c0 tx=0x7f818400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.435+0000 7f8190c00700 1 -- 172.21.15.106:0/2827714140 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818400cbd0 con 0x7f8194075650 2024-04-28T12:50:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.435+0000 7f8190c00700 1 -- 172.21.15.106:0/2827714140 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818400ee60 con 0x7f8194075650 2024-04-28T12:50:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.435+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81941a1120 con 0x7f8194075650 2024-04-28T12:50:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.435+0000 7f8190c00700 1 -- 172.21.15.106:0/2827714140 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81840055c0 con 0x7f8194075650 2024-04-28T12:50:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.435+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81941a89a0 con 0x7f8194075650 2024-04-28T12:50:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.436+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8194063b30 con 0x7f8194075650 2024-04-28T12:50:05.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.439+0000 7f8190c00700 1 -- 172.21.15.106:0/2827714140 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8184004750 con 0x7f8194075650 2024-04-28T12:50:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.440+0000 7f8190c00700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81780a3cd0 0x7f81780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.440+0000 7f8190c00700 1 -- 172.21.15.106:0/2827714140 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f81840d0f90 con 0x7f8194075650 2024-04-28T12:50:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.440+0000 7f8193400700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81780a3cd0 0x7f81780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.440+0000 7f8190c00700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] conn(0x7f81780aedb0 0x7f81780b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.440+0000 7f8190c00700 1 -- 172.21.15.106:0/2827714140 --> [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f818401ee50 con 0x7f81780aedb0 2024-04-28T12:50:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.440+0000 7f8190c00700 1 -- 172.21.15.106:0/2827714140 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f81840d1340 con 0x7f8194075650 2024-04-28T12:50:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.441+0000 7f8199000700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] conn(0x7f81780aedb0 0x7f81780b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.441+0000 7f8193400700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81780a3cd0 0x7f81780a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8194103000 tx=0x7f818000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.441+0000 7f8199000700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] conn(0x7f81780aedb0 0x7f81780b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.442+0000 7f8190c00700 1 -- 172.21.15.106:0/2827714140 <== osd.3 v2:172.21.15.106:6810/3527734792 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f818401ee50 con 0x7f81780aedb0 2024-04-28T12:50:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.460+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 --> [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8194063fb0 con 0x7f81780aedb0 2024-04-28T12:50:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.461+0000 7f8190c00700 1 -- 172.21.15.106:0/2827714140 <== osd.3 v2:172.21.15.106:6810/3527734792 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8194063fb0 con 0x7f81780aedb0 2024-04-28T12:50:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.461+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] conn(0x7f81780aedb0 msgr2=0x7f81780b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.461+0000 7f819b000700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] conn(0x7f81780aedb0 0x7f81780b1250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.462+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81780a3cd0 msgr2=0x7f81780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.462+0000 7f819b000700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f81780a3cd0 0x7f81780a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8194103000 tx=0x7f818000b260 comp rx=0 tx=0).stop 2024-04-28T12:50:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.462+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8194075650 msgr2=0x7f819419be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.462+0000 7f819b000700 1 --2- 172.21.15.106:0/2827714140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8194075650 0x7f819419be20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f818400e9c0 tx=0x7f818400ecd0 comp rx=0 tx=0).stop 2024-04-28T12:50:05.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.462+0000 7f8199000700 1 -- 172.21.15.106:0/2827714140 reap_dead start 2024-04-28T12:50:05.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.462+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 shutdown_connections 2024-04-28T12:50:05.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.462+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 >> 172.21.15.106:0/2827714140 conn(0x7f819407a5d0 msgr2=0x7f81941019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:05.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.462+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 shutdown_connections 2024-04-28T12:50:05.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:05.462+0000 7f819b000700 1 -- 172.21.15.106:0/2827714140 wait complete. 2024-04-28T12:50:05.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:05 smithi171 ceph-mon[63127]: pgmap v355: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:05.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:05 smithi129 ceph-mon[62418]: pgmap v355: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:05.842 INFO:teuthology.orchestra.run.smithi106.stdout:55834574930 2024-04-28T12:50:05.843 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-28T12:50:07.289 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:07 smithi106 ceph-mon[28081]: pgmap v356: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:07.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:07 smithi171 ceph-mon[63127]: pgmap v356: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:07.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:07 smithi129 ceph-mon[62418]: pgmap v356: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.036+0000 7fb9f9000700 1 -- 172.21.15.106:0/1982879633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f406b130 msgr2=0x7fb9f406b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.036+0000 7fb9f9000700 1 --2- 172.21.15.106:0/1982879633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f406b130 0x7fb9f406b590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009990 tx=0x7fb9e0009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.036+0000 7fb9f9000700 1 -- 172.21.15.106:0/1982879633 shutdown_connections 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.036+0000 7fb9f9000700 1 --2- 172.21.15.106:0/1982879633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f406bad0 0x7fb9f41102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.036+0000 7fb9f9000700 1 --2- 172.21.15.106:0/1982879633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9f41094a0 0x7fb9f4109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.036+0000 7fb9f9000700 1 --2- 172.21.15.106:0/1982879633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f406b130 0x7fb9f406b590 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.036+0000 7fb9f9000700 1 -- 172.21.15.106:0/1982879633 >> 172.21.15.106:0/1982879633 conn(0x7fb9f40792d0 msgr2=0x7fb9f40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.036+0000 7fb9f9000700 1 -- 172.21.15.106:0/1982879633 shutdown_connections 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.036+0000 7fb9f9000700 1 -- 172.21.15.106:0/1982879633 wait complete. 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.037+0000 7fb9f9000700 1 Processor -- start 2024-04-28T12:50:07.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.037+0000 7fb9f9000700 1 -- start start 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f9000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f406b130 0x7fb9f410a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f9000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f406bad0 0x7fb9f410a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f9000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9f41094a0 0x7fb9f410ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f9000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4113600 con 0x7fb9f406b130 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f9000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4113480 con 0x7fb9f406bad0 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f9000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4113780 con 0x7fb9f41094a0 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f1e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f406b130 0x7fb9f410a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f1e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f406b130 0x7fb9f410a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41658/0 (socket says 172.21.15.106:41658) 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f1e00700 1 -- 172.21.15.106:0/834210925 learned_addr learned my addr 172.21.15.106:0/834210925 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f1400700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f406bad0 0x7fb9f410a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.038+0000 7fb9f2800700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9f41094a0 0x7fb9f410ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:07.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9f1e00700 1 -- 172.21.15.106:0/834210925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9f41094a0 msgr2=0x7fb9f410ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9f1e00700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9f41094a0 0x7fb9f410ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9f1e00700 1 -- 172.21.15.106:0/834210925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f406bad0 msgr2=0x7fb9f410a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9f1e00700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9f406bad0 0x7fb9f410a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9f1e00700 1 -- 172.21.15.106:0/834210925 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f41ac400 con 0x7fb9f406b130 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9f2800700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9f41094a0 0x7fb9f410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9f1e00700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f406b130 0x7fb9f410a360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400e9c0 tx=0x7fb9e400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9eaa00700 1 -- 172.21.15.106:0/834210925 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e400cbd0 con 0x7fb9f406b130 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9eaa00700 1 -- 172.21.15.106:0/834210925 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9e400cd70 con 0x7fb9f406b130 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.039+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f41ac6f0 con 0x7fb9f406b130 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.040+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f41acc30 con 0x7fb9f406b130 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.040+0000 7fb9eaa00700 1 -- 172.21.15.106:0/834210925 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4005680 con 0x7fb9f406b130 2024-04-28T12:50:07.613 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.041+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9d4001060 con 0x7fb9f406b130 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.043+0000 7fb9eaa00700 1 -- 172.21.15.106:0/834210925 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb9e4004750 con 0x7fb9f406b130 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.044+0000 7fb9eaa00700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb9d80a3d50 0x7fb9d80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.045+0000 7fb9eaa00700 1 -- 172.21.15.106:0/834210925 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fb9e40d0d60 con 0x7fb9f406b130 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.045+0000 7fb9eaa00700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] conn(0x7fb9d80aee30 0x7fb9d80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.045+0000 7fb9eaa00700 1 -- 172.21.15.106:0/834210925 --> [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9e4017e50 con 0x7fb9d80aee30 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.045+0000 7fb9eaa00700 1 -- 172.21.15.106:0/834210925 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fb9e40d1110 con 0x7fb9f406b130 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.045+0000 7fb9f1400700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb9d80a3d50 0x7fb9d80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.045+0000 7fb9f2800700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] conn(0x7fb9d80aee30 0x7fb9d80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.046+0000 7fb9f1400700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb9d80a3d50 0x7fb9d80a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9f410bb20 tx=0x7fb9e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.046+0000 7fb9f2800700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] conn(0x7fb9d80aee30 0x7fb9d80b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.047+0000 7fb9eaa00700 1 -- 172.21.15.106:0/834210925 <== osd.5 v2:172.21.15.171:6808/1286181714 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9e4017e50 con 0x7fb9d80aee30 2024-04-28T12:50:07.614 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.065+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 --> [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9d4002e90 con 0x7fb9d80aee30 2024-04-28T12:50:07.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.065+0000 7fb9eaa00700 1 -- 172.21.15.106:0/834210925 <== osd.5 v2:172.21.15.171:6808/1286181714 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb9d4002e90 con 0x7fb9d80aee30 2024-04-28T12:50:07.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.066+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 >> [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] conn(0x7fb9d80aee30 msgr2=0x7fb9d80b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:07.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.066+0000 7fb9f9000700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] conn(0x7fb9d80aee30 0x7fb9d80b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:07.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.066+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb9d80a3d50 msgr2=0x7fb9d80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:07.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.066+0000 7fb9f9000700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb9d80a3d50 0x7fb9d80a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb9f410bb20 tx=0x7fb9e0005fd0 comp rx=0 tx=0).stop 2024-04-28T12:50:07.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.066+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f406b130 msgr2=0x7fb9f410a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:07.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.066+0000 7fb9f9000700 1 --2- 172.21.15.106:0/834210925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f406b130 0x7fb9f410a360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400e9c0 tx=0x7fb9e400ecd0 comp rx=0 tx=0).stop 2024-04-28T12:50:07.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.066+0000 7fb9f2800700 1 -- 172.21.15.106:0/834210925 reap_dead start 2024-04-28T12:50:07.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.066+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 shutdown_connections 2024-04-28T12:50:07.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.066+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 >> 172.21.15.106:0/834210925 conn(0x7fb9f40792d0 msgr2=0x7fb9f410e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:07.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.067+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 shutdown_connections 2024-04-28T12:50:07.616 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.067+0000 7fb9f9000700 1 -- 172.21.15.106:0/834210925 wait complete. 2024-04-28T12:50:08.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.676+0000 7f748f600700 1 -- 172.21.15.106:0/3315223742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f748810a410 msgr2=0x7f7488116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:08.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.676+0000 7f748f600700 1 --2- 172.21.15.106:0/3315223742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f748810a410 0x7f7488116920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f748000b0a0 tx=0x7f748000b3b0 comp rx=0 tx=0).stop 2024-04-28T12:50:08.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.676+0000 7f748f600700 1 -- 172.21.15.106:0/3315223742 shutdown_connections 2024-04-28T12:50:08.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.676+0000 7f748f600700 1 --2- 172.21.15.106:0/3315223742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f748810a410 0x7f7488116920 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:08.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.676+0000 7f748f600700 1 --2- 172.21.15.106:0/3315223742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f748810fa50 0x7f748810fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:08.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.676+0000 7f748f600700 1 --2- 172.21.15.106:0/3315223742 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7488109a50 0x7f7488109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:08.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.676+0000 7f748f600700 1 -- 172.21.15.106:0/3315223742 >> 172.21.15.106:0/3315223742 conn(0x7f7488076bf0 msgr2=0x7f7488108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.676+0000 7f748f600700 1 -- 172.21.15.106:0/3315223742 shutdown_connections 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.676+0000 7f748f600700 1 -- 172.21.15.106:0/3315223742 wait complete. 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.677+0000 7f748f600700 1 Processor -- start 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.677+0000 7f748f600700 1 -- start start 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748f600700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7488109a50 0x7f74881a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748f600700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f748810a410 0x7f74881a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748f600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f748810fa50 0x7f74881a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748f600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7488119c30 con 0x7f748810fa50 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748f600700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7488119ab0 con 0x7f7488109a50 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748f600700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7488119db0 con 0x7f748810a410 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748d600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f748810fa50 0x7f74881a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748cc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7488109a50 0x7f74881a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f7487e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f748810a410 0x7f74881a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748d600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f748810fa50 0x7f74881a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41712/0 (socket says 172.21.15.106:41712) 2024-04-28T12:50:08.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748d600700 1 -- 172.21.15.106:0/978646411 learned_addr learned my addr 172.21.15.106:0/978646411 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748cc00700 1 -- 172.21.15.106:0/978646411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f748810a410 msgr2=0x7f74881a05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748cc00700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f748810a410 0x7f74881a05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748cc00700 1 -- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f748810fa50 msgr2=0x7f74881a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748cc00700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f748810fa50 0x7f74881a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748cc00700 1 -- 172.21.15.106:0/978646411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74881a5070 con 0x7f7488109a50 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.678+0000 7f748d600700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f748810fa50 0x7f74881a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.679+0000 7f748cc00700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7488109a50 0x7f74881a0060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f747c00e9f0 tx=0x7f747c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.679+0000 7f7485600700 1 -- 172.21.15.106:0/978646411 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c00c9c0 con 0x7f7488109a50 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.679+0000 7f7485600700 1 -- 172.21.15.106:0/978646411 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f747c00cb60 con 0x7f7488109a50 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.679+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74881a5360 con 0x7f7488109a50 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.679+0000 7f7485600700 1 -- 172.21.15.106:0/978646411 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c018680 con 0x7f7488109a50 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.679+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74881acbe0 con 0x7f7488109a50 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.680+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7488063bb0 con 0x7f7488109a50 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.682+0000 7f7485600700 1 -- 172.21.15.106:0/978646411 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f747c010430 con 0x7f7488109a50 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.684+0000 7f7485600700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f74700a3cd0 0x7f74700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:08.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.684+0000 7f7485600700 1 -- 172.21.15.106:0/978646411 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f747c0d0fa0 con 0x7f7488109a50 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.684+0000 7f7487e00700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f74700a3cd0 0x7f74700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.684+0000 7f7485600700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] conn(0x7f74700aedb0 0x7f74700b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.684+0000 7f7485600700 1 -- 172.21.15.106:0/978646411 --> [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f747c019ec0 con 0x7f74700aedb0 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.684+0000 7f7485600700 1 -- 172.21.15.106:0/978646411 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f747c0d1350 con 0x7f7488109a50 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.684+0000 7f748d600700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] conn(0x7f74700aedb0 0x7f74700b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.685+0000 7f7487e00700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f74700a3cd0 0x7f74700a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7488078a70 tx=0x7f74740073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.685+0000 7f748d600700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] conn(0x7f74700aedb0 0x7f74700b1250 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.686+0000 7f7485600700 1 -- 172.21.15.106:0/978646411 <== osd.6 v2:172.21.15.106:6818/1789189109 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f747c019ec0 con 0x7f74700aedb0 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.704+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 --> [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7488064030 con 0x7f74700aedb0 2024-04-28T12:50:08.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.704+0000 7f7485600700 1 -- 172.21.15.106:0/978646411 <== osd.6 v2:172.21.15.106:6818/1789189109 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7488064030 con 0x7f74700aedb0 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] conn(0x7f74700aedb0 msgr2=0x7f74700b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] conn(0x7f74700aedb0 0x7f74700b1250 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f74700a3cd0 msgr2=0x7f74700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f74700a3cd0 0x7f74700a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7488078a70 tx=0x7f74740073d0 comp rx=0 tx=0).stop 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7488109a50 msgr2=0x7f74881a0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 --2- 172.21.15.106:0/978646411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7488109a50 0x7f74881a0060 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f747c00e9f0 tx=0x7f747c00ed00 comp rx=0 tx=0).stop 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748d600700 1 -- 172.21.15.106:0/978646411 reap_dead start 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 shutdown_connections 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 >> 172.21.15.106:0/978646411 conn(0x7f7488076bf0 msgr2=0x7f7488077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 shutdown_connections 2024-04-28T12:50:08.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.705+0000 7f748f600700 1 -- 172.21.15.106:0/978646411 wait complete. 2024-04-28T12:50:09.203 INFO:teuthology.orchestra.run.smithi106.stdout:197568495673 2024-04-28T12:50:09.203 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-28T12:50:09.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:09 smithi171 ceph-mon[63127]: pgmap v357: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:09.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:09 smithi129 ceph-mon[62418]: pgmap v357: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:09.516 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:09 smithi106 ceph-mon[28081]: pgmap v357: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.415+0000 7f2923000700 1 -- 172.21.15.106:0/1323828559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f291c06c490 msgr2=0x7f291c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.415+0000 7f2923000700 1 --2- 172.21.15.106:0/1323828559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f291c06c490 0x7f291c110300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f291400b0a0 tx=0x7f291400b3b0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.415+0000 7f2923000700 1 -- 172.21.15.106:0/1323828559 shutdown_connections 2024-04-28T12:50:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.415+0000 7f2923000700 1 --2- 172.21.15.106:0/1323828559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f291c06c490 0x7f291c110300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.415+0000 7f2923000700 1 --2- 172.21.15.106:0/1323828559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c1094c0 0x7f291c1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.415+0000 7f2923000700 1 --2- 172.21.15.106:0/1323828559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f291c06baf0 0x7f291c06bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.415+0000 7f2923000700 1 -- 172.21.15.106:0/1323828559 >> 172.21.15.106:0/1323828559 conn(0x7f291c0792d0 msgr2=0x7f291c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.415+0000 7f2923000700 1 -- 172.21.15.106:0/1323828559 shutdown_connections 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.415+0000 7f2923000700 1 -- 172.21.15.106:0/1323828559 wait complete. 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.416+0000 7f2923000700 1 Processor -- start 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.416+0000 7f2923000700 1 -- start start 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f2923000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c06baf0 0x7f291c1027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f2923000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f291c06c490 0x7f291c102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f2923000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f291c1094c0 0x7f291c103250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f2923000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f291c1135e0 con 0x7f291c1094c0 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f2923000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f291c113460 con 0x7f291c06c490 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f2923000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f291c113760 con 0x7f291c06baf0 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f2921000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f291c1094c0 0x7f291c103250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f291b400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f291c06c490 0x7f291c102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f2921000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f291c1094c0 0x7f291c103250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41720/0 (socket says 172.21.15.106:41720) 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f2921000700 1 -- 172.21.15.106:0/2787709373 learned_addr learned my addr 172.21.15.106:0/2787709373 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.417+0000 7f291be00700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c06baf0 0x7f291c1027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f291b400700 1 -- 172.21.15.106:0/2787709373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c06baf0 msgr2=0x7f291c1027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f291b400700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f291c06baf0 0x7f291c1027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f291b400700 1 -- 172.21.15.106:0/2787709373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f291c1094c0 msgr2=0x7f291c103250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f291b400700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f291c1094c0 0x7f291c103250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f291b400700 1 -- 172.21.15.106:0/2787709373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c1a8000 con 0x7f291c06c490 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f2921000700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f291c1094c0 0x7f291c103250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f291b400700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f291c06c490 0x7f291c102d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f290800e9f0 tx=0x7f290800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f2918c00700 1 -- 172.21.15.106:0/2787709373 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290800c9c0 con 0x7f291c06c490 2024-04-28T12:50:10.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f2918c00700 1 -- 172.21.15.106:0/2787709373 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f290800cb60 con 0x7f291c06c490 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f291c1a82f0 con 0x7f291c06c490 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f2918c00700 1 -- 172.21.15.106:0/2787709373 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908018680 con 0x7f291c06c490 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.418+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291c1a8830 con 0x7f291c06c490 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.419+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f291c063af0 con 0x7f291c06c490 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.422+0000 7f2918c00700 1 -- 172.21.15.106:0/2787709373 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2908010430 con 0x7f291c06c490 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.423+0000 7f2918c00700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f29000a3cd0 0x7f29000a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.424+0000 7f2918c00700 1 -- 172.21.15.106:0/2787709373 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f29080d0fe0 con 0x7f291c06c490 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.424+0000 7f291be00700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f29000a3cd0 0x7f29000a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.424+0000 7f2918c00700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] conn(0x7f29000aedb0 0x7f29000b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.424+0000 7f2918c00700 1 -- 172.21.15.106:0/2787709373 --> [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2908019ec0 con 0x7f29000aedb0 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.424+0000 7f2918c00700 1 -- 172.21.15.106:0/2787709373 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f29080d1410 con 0x7f291c06c490 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.424+0000 7f2921000700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] conn(0x7f29000aedb0 0x7f29000b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.424+0000 7f291be00700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f29000a3cd0 0x7f29000a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f290c009be0 tx=0x7f290c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.425+0000 7f2921000700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] conn(0x7f29000aedb0 0x7f29000b1250 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:10.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.426+0000 7f2918c00700 1 -- 172.21.15.106:0/2787709373 <== osd.4 v2:172.21.15.129:6808/2253162285 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2908019ec0 con 0x7f29000aedb0 2024-04-28T12:50:10.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.444+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 --> [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f291c063f70 con 0x7f29000aedb0 2024-04-28T12:50:10.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.444+0000 7f2918c00700 1 -- 172.21.15.106:0/2787709373 <== osd.4 v2:172.21.15.129:6808/2253162285 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f291c063f70 con 0x7f29000aedb0 2024-04-28T12:50:10.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 >> [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] conn(0x7f29000aedb0 msgr2=0x7f29000b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2923000700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] conn(0x7f29000aedb0 0x7f29000b1250 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f29000a3cd0 msgr2=0x7f29000a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2923000700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f29000a3cd0 0x7f29000a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f290c009be0 tx=0x7f290c009260 comp rx=0 tx=0).stop 2024-04-28T12:50:10.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f291c06c490 msgr2=0x7f291c102d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2923000700 1 --2- 172.21.15.106:0/2787709373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f291c06c490 0x7f291c102d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f290800e9f0 tx=0x7f290800ed00 comp rx=0 tx=0).stop 2024-04-28T12:50:10.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2921000700 1 -- 172.21.15.106:0/2787709373 reap_dead start 2024-04-28T12:50:10.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 shutdown_connections 2024-04-28T12:50:10.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 >> 172.21.15.106:0/2787709373 conn(0x7f291c0792d0 msgr2=0x7f291c1008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:10.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.445+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 shutdown_connections 2024-04-28T12:50:10.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:08.446+0000 7f2923000700 1 -- 172.21.15.106:0/2787709373 wait complete. 2024-04-28T12:50:10.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.131+0000 7fa9a3800700 1 -- 172.21.15.106:0/1454826488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa99c103e30 msgr2=0x7fa99c110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.131+0000 7fa9a3800700 1 --2- 172.21.15.106:0/1454826488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa99c103e30 0x7fa99c110290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa994009f90 tx=0x7fa99400b330 comp rx=0 tx=0).stop 2024-04-28T12:50:10.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.131+0000 7fa9a3800700 1 -- 172.21.15.106:0/1454826488 shutdown_connections 2024-04-28T12:50:10.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.131+0000 7fa9a3800700 1 --2- 172.21.15.106:0/1454826488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa99c103e30 0x7fa99c110290 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.131+0000 7fa9a3800700 1 --2- 172.21.15.106:0/1454826488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa99c1094c0 0x7fa99c1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.131+0000 7fa9a3800700 1 --2- 172.21.15.106:0/1454826488 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa99c103470 0x7fa99c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.131+0000 7fa9a3800700 1 -- 172.21.15.106:0/1454826488 >> 172.21.15.106:0/1454826488 conn(0x7fa99c07a580 msgr2=0x7fa99c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:10.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.131+0000 7fa9a3800700 1 -- 172.21.15.106:0/1454826488 shutdown_connections 2024-04-28T12:50:10.700 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.131+0000 7fa9a3800700 1 -- 172.21.15.106:0/1454826488 wait complete. 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.132+0000 7fa9a3800700 1 Processor -- start 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.132+0000 7fa9a3800700 1 -- start start 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa9a3800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa99c103470 0x7fa99c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa9a3800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa99c103e30 0x7fa99c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa9a3800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa99c1094c0 0x7fa99c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa9a3800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c113610 con 0x7fa99c103e30 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa9a3800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c113490 con 0x7fa99c1094c0 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa9a3800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa99c113790 con 0x7fa99c103470 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa99be00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa99c103e30 0x7fa99c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa9a1800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa99c1094c0 0x7fa99c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa99be00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa99c103e30 0x7fa99c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41682/0 (socket says 172.21.15.106:41682) 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa99be00700 1 -- 172.21.15.106:0/1341391582 learned_addr learned my addr 172.21.15.106:0/1341391582 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.133+0000 7fa9a0e00700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa99c103470 0x7fa99c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.701 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.134+0000 7fa99be00700 1 -- 172.21.15.106:0/1341391582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa99c103470 msgr2=0x7fa99c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.134+0000 7fa99be00700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa99c103470 0x7fa99c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.134+0000 7fa99be00700 1 -- 172.21.15.106:0/1341391582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa99c1094c0 msgr2=0x7fa99c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.134+0000 7fa99be00700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa99c1094c0 0x7fa99c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.134+0000 7fa99be00700 1 -- 172.21.15.106:0/1341391582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99c19a070 con 0x7fa99c103e30 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.134+0000 7fa99be00700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa99c103e30 0x7fa99c19fc90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa98800e9a0 tx=0x7fa98800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.135+0000 7fa999600700 1 -- 172.21.15.106:0/1341391582 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98800cbd0 con 0x7fa99c103e30 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.135+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99c19a360 con 0x7fa99c103e30 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.135+0000 7fa999600700 1 -- 172.21.15.106:0/1341391582 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa98800cd70 con 0x7fa99c103e30 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.135+0000 7fa999600700 1 -- 172.21.15.106:0/1341391582 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988018680 con 0x7fa99c103e30 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.135+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99c1a6620 con 0x7fa99c103e30 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.136+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa99c063bb0 con 0x7fa99c103e30 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.139+0000 7fa999600700 1 -- 172.21.15.106:0/1341391582 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa988004020 con 0x7fa99c103e30 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.141+0000 7fa999600700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa9840a3cd0 0x7fa9840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.141+0000 7fa999600700 1 -- 172.21.15.106:0/1341391582 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fa9880d0cb0 con 0x7fa99c103e30 2024-04-28T12:50:10.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.141+0000 7fa9a0e00700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa9840a3cd0 0x7fa9840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.141+0000 7fa999600700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] conn(0x7fa9840aedb0 0x7fa9840b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.141+0000 7fa999600700 1 -- 172.21.15.106:0/1341391582 --> [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa988010ab0 con 0x7fa9840aedb0 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.141+0000 7fa999600700 1 -- 172.21.15.106:0/1341391582 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fa9880d1060 con 0x7fa99c103e30 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.141+0000 7fa9a1800700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] conn(0x7fa9840aedb0 0x7fa9840b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.142+0000 7fa9a0e00700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa9840a3cd0 0x7fa9840a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa99c19aa50 tx=0x7fa990009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.142+0000 7fa9a1800700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] conn(0x7fa9840aedb0 0x7fa9840b1250 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.143+0000 7fa999600700 1 -- 172.21.15.106:0/1341391582 <== osd.7 v2:172.21.15.129:6816/3741780752 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa988010ab0 con 0x7fa9840aedb0 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.161+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 --> [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa99c064030 con 0x7fa9840aedb0 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.162+0000 7fa999600700 1 -- 172.21.15.106:0/1341391582 <== osd.7 v2:172.21.15.129:6816/3741780752 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa99c064030 con 0x7fa9840aedb0 2024-04-28T12:50:10.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.162+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 >> [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] conn(0x7fa9840aedb0 msgr2=0x7fa9840b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.162+0000 7fa9a3800700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] conn(0x7fa9840aedb0 0x7fa9840b1250 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.163+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa9840a3cd0 msgr2=0x7fa9840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.163+0000 7fa9a3800700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa9840a3cd0 0x7fa9840a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa99c19aa50 tx=0x7fa990009310 comp rx=0 tx=0).stop 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.163+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa99c103e30 msgr2=0x7fa99c19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.163+0000 7fa9a3800700 1 --2- 172.21.15.106:0/1341391582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa99c103e30 0x7fa99c19fc90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa98800e9a0 tx=0x7fa98800ed60 comp rx=0 tx=0).stop 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.163+0000 7fa9a1800700 1 -- 172.21.15.106:0/1341391582 reap_dead start 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.163+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 shutdown_connections 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.163+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 >> 172.21.15.106:0/1341391582 conn(0x7fa99c07a580 msgr2=0x7fa99c102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.163+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 shutdown_connections 2024-04-28T12:50:10.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:07.163+0000 7fa9a3800700 1 -- 172.21.15.106:0/1341391582 wait complete. 2024-04-28T12:50:11.397 INFO:teuthology.orchestra.run.smithi106.stdout:163208757310 2024-04-28T12:50:11.398 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-28T12:50:11.425 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:11 smithi106 ceph-mon[28081]: pgmap v358: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:11.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:11 smithi171 ceph-mon[63127]: pgmap v358: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:11.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:11 smithi129 ceph-mon[62418]: pgmap v358: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-28T12:50:11.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.596+0000 7fcd82e00700 1 -- 172.21.15.106:0/3421737355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd7c06bad0 msgr2=0x7fcd7c1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:11.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.596+0000 7fcd82e00700 1 --2- 172.21.15.106:0/3421737355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd7c06bad0 0x7fcd7c1102e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400b0a0 tx=0x7fcd7400b3b0 comp rx=0 tx=0).stop 2024-04-28T12:50:11.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.596+0000 7fcd82e00700 1 -- 172.21.15.106:0/3421737355 shutdown_connections 2024-04-28T12:50:11.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.596+0000 7fcd82e00700 1 --2- 172.21.15.106:0/3421737355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd7c06bad0 0x7fcd7c1102e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:11.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.596+0000 7fcd82e00700 1 --2- 172.21.15.106:0/3421737355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd7c1094a0 0x7fcd7c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:11.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.596+0000 7fcd82e00700 1 --2- 172.21.15.106:0/3421737355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c06b130 0x7fcd7c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:11.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.596+0000 7fcd82e00700 1 -- 172.21.15.106:0/3421737355 >> 172.21.15.106:0/3421737355 conn(0x7fcd7c0792d0 msgr2=0x7fcd7c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:11.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.596+0000 7fcd82e00700 1 -- 172.21.15.106:0/3421737355 shutdown_connections 2024-04-28T12:50:11.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.596+0000 7fcd82e00700 1 -- 172.21.15.106:0/3421737355 wait complete. 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.597+0000 7fcd82e00700 1 Processor -- start 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.597+0000 7fcd82e00700 1 -- start start 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd82e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd7c06b130 0x7fcd7c1a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd82e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c06bad0 0x7fcd7c1a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd82e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd7c1094a0 0x7fcd7c19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd82e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c113610 con 0x7fcd7c06bad0 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd82e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c113490 con 0x7fcd7c06b130 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd82e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c113790 con 0x7fcd7c1094a0 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd7b400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c06bad0 0x7fcd7c1a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd7be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd7c06b130 0x7fcd7c1a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd7be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd7c06b130 0x7fcd7c1a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:47244/0 (socket says 172.21.15.106:47244) 2024-04-28T12:50:11.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd7b400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c06bad0 0x7fcd7c1a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44242/0 (socket says 172.21.15.106:44242) 2024-04-28T12:50:11.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd7be00700 1 -- 172.21.15.106:0/1125908331 learned_addr learned my addr 172.21.15.106:0/1125908331 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:11.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.598+0000 7fcd80e00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd7c1094a0 0x7fcd7c19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.599+0000 7fcd7be00700 1 -- 172.21.15.106:0/1125908331 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd7c1094a0 msgr2=0x7fcd7c19bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.599+0000 7fcd7be00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcd7c1094a0 0x7fcd7c19bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.599+0000 7fcd7be00700 1 -- 172.21.15.106:0/1125908331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c06bad0 msgr2=0x7fcd7c1a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.599+0000 7fcd7be00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd7c06bad0 0x7fcd7c1a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.599+0000 7fcd7be00700 1 -- 172.21.15.106:0/1125908331 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c19c2a0 con 0x7fcd7c06b130 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.599+0000 7fcd7be00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd7c06b130 0x7fcd7c1a1930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e9c0 tx=0x7fcd6c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.600+0000 7fcd78c00700 1 -- 172.21.15.106:0/1125908331 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00cbd0 con 0x7fcd7c06b130 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.600+0000 7fcd78c00700 1 -- 172.21.15.106:0/1125908331 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd6c00ee60 con 0x7fcd7c06b130 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.600+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c19c590 con 0x7fcd7c06b130 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.600+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c1a8800 con 0x7fcd7c06b130 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.600+0000 7fcd78c00700 1 -- 172.21.15.106:0/1125908331 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0055c0 con 0x7fcd7c06b130 2024-04-28T12:50:11.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.601+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd7c063b70 con 0x7fcd7c06b130 2024-04-28T12:50:11.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.603+0000 7fcd78c00700 1 -- 172.21.15.106:0/1125908331 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fcd6c004750 con 0x7fcd7c06b130 2024-04-28T12:50:11.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.604+0000 7fcd78c00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcd600a3d50 0x7fcd600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:11.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.604+0000 7fcd78c00700 1 -- 172.21.15.106:0/1125908331 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fcd6c0d0ec0 con 0x7fcd7c06b130 2024-04-28T12:50:11.601 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.605+0000 7fcd7b400700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcd600a3d50 0x7fcd600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:11.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.605+0000 7fcd78c00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] conn(0x7fcd600aee30 0x7fcd600b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:11.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.605+0000 7fcd78c00700 1 -- 172.21.15.106:0/1125908331 --> [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd600b19a0 con 0x7fcd600aee30 2024-04-28T12:50:11.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.605+0000 7fcd78c00700 1 -- 172.21.15.106:0/1125908331 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fcd6c0d1270 con 0x7fcd7c06b130 2024-04-28T12:50:11.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.605+0000 7fcd80e00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] conn(0x7fcd600aee30 0x7fcd600b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:11.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.606+0000 7fcd7b400700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcd600a3d50 0x7fcd600a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c079840 tx=0x7fcd68009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:11.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.606+0000 7fcd80e00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] conn(0x7fcd600aee30 0x7fcd600b12d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:11.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.607+0000 7fcd78c00700 1 -- 172.21.15.106:0/1125908331 <== osd.2 v2:172.21.15.171:6800/1763026525 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd600b19a0 con 0x7fcd600aee30 2024-04-28T12:50:11.621 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.625+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 --> [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd7c063f70 con 0x7fcd600aee30 2024-04-28T12:50:11.622 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.625+0000 7fcd78c00700 1 -- 172.21.15.106:0/1125908331 <== osd.2 v2:172.21.15.171:6800/1763026525 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcd7c063f70 con 0x7fcd600aee30 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.626+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 >> [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] conn(0x7fcd600aee30 msgr2=0x7fcd600b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.626+0000 7fcd82e00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] conn(0x7fcd600aee30 0x7fcd600b12d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.626+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcd600a3d50 msgr2=0x7fcd600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.626+0000 7fcd82e00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcd600a3d50 0x7fcd600a6210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c079840 tx=0x7fcd68009260 comp rx=0 tx=0).stop 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.626+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd7c06b130 msgr2=0x7fcd7c1a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.626+0000 7fcd82e00700 1 --2- 172.21.15.106:0/1125908331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd7c06b130 0x7fcd7c1a1930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e9c0 tx=0x7fcd6c00ecd0 comp rx=0 tx=0).stop 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.626+0000 7fcd80e00700 1 -- 172.21.15.106:0/1125908331 reap_dead start 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.627+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 shutdown_connections 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.627+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 >> 172.21.15.106:0/1125908331 conn(0x7fcd7c0792d0 msgr2=0x7fcd7c10e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:11.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.627+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 shutdown_connections 2024-04-28T12:50:11.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:11.627+0000 7fcd82e00700 1 -- 172.21.15.106:0/1125908331 wait complete. 2024-04-28T12:50:11.699 INFO:teuthology.orchestra.run.smithi106.stdout:309237645343 2024-04-28T12:50:11.699 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-28T12:50:12.020 INFO:teuthology.orchestra.run.smithi106.stdout:343597383706 2024-04-28T12:50:12.020 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-28T12:50:13.390 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:13 smithi106 ceph-mon[28081]: pgmap v359: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:13.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:13 smithi171 ceph-mon[63127]: pgmap v359: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:13.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:13 smithi129 ceph-mon[62418]: pgmap v359: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:13.851 INFO:teuthology.orchestra.run.smithi106.stdout:77309411409 2024-04-28T12:50:13.851 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-28T12:50:14.731 INFO:teuthology.orchestra.run.smithi106.stdout:141733920836 2024-04-28T12:50:14.731 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-28T12:50:14.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.929+0000 7f3dce800700 1 -- 172.21.15.106:0/1429474550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dc8075650 msgr2=0x7f3dc8075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:14.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.929+0000 7f3dce800700 1 --2- 172.21.15.106:0/1429474550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dc8075650 0x7f3dc8075a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009990 tx=0x7f3db8009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:14.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.929+0000 7f3dce800700 1 -- 172.21.15.106:0/1429474550 shutdown_connections 2024-04-28T12:50:14.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.929+0000 7f3dce800700 1 --2- 172.21.15.106:0/1429474550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dc810fe80 0x7f3dc8112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:14.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.929+0000 7f3dce800700 1 --2- 172.21.15.106:0/1429474550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dc8075650 0x7f3dc8075a30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:14.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.929+0000 7f3dce800700 1 --2- 172.21.15.106:0/1429474550 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3dc8075f70 0x7f3dc810f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:14.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.929+0000 7f3dce800700 1 -- 172.21.15.106:0/1429474550 >> 172.21.15.106:0/1429474550 conn(0x7f3dc807a500 msgr2=0x7f3dc807a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:14.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.929+0000 7f3dce800700 1 -- 172.21.15.106:0/1429474550 shutdown_connections 2024-04-28T12:50:14.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.929+0000 7f3dce800700 1 -- 172.21.15.106:0/1429474550 wait complete. 2024-04-28T12:50:14.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.931+0000 7f3dce800700 1 Processor -- start 2024-04-28T12:50:14.927 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.931+0000 7f3dce800700 1 -- start start 2024-04-28T12:50:14.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.931+0000 7f3dce800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dc8075650 0x7f3dc819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:14.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.931+0000 7f3dce800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dc8075f70 0x7f3dc819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:14.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.931+0000 7f3dce800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3dc810fe80 0x7f3dc81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:14.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.931+0000 7f3dce800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8115890 con 0x7f3dc8075650 2024-04-28T12:50:14.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.931+0000 7f3dce800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8115710 con 0x7f3dc8075f70 2024-04-28T12:50:14.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.931+0000 7f3dce800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8115a10 con 0x7f3dc810fe80 2024-04-28T12:50:14.928 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc7400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dc8075650 0x7f3dc819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:14.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc6a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dc8075f70 0x7f3dc819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:14.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc7e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3dc810fe80 0x7f3dc81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:14.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc6a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dc8075f70 0x7f3dc819c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:47256/0 (socket says 172.21.15.106:47256) 2024-04-28T12:50:14.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc7400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dc8075650 0x7f3dc819bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44258/0 (socket says 172.21.15.106:44258) 2024-04-28T12:50:14.929 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc6a00700 1 -- 172.21.15.106:0/1359599560 learned_addr learned my addr 172.21.15.106:0/1359599560 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc7e00700 1 -- 172.21.15.106:0/1359599560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dc8075f70 msgr2=0x7f3dc819c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc7e00700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dc8075f70 0x7f3dc819c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc7e00700 1 -- 172.21.15.106:0/1359599560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dc8075650 msgr2=0x7f3dc819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc7e00700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dc8075650 0x7f3dc819bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc7e00700 1 -- 172.21.15.106:0/1359599560 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc81a0d80 con 0x7f3dc810fe80 2024-04-28T12:50:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc6a00700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3dc8075f70 0x7f3dc819c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.932+0000 7f3dc7400700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3dc8075650 0x7f3dc819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.933+0000 7f3dc7e00700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3dc810fe80 0x7f3dc81a0640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000ef90 tx=0x7f3dc000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.933+0000 7f3dbfe00700 1 -- 172.21.15.106:0/1359599560 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00092e0 con 0x7f3dc810fe80 2024-04-28T12:50:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.933+0000 7f3dbfe00700 1 -- 172.21.15.106:0/1359599560 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc0004bb0 con 0x7f3dc810fe80 2024-04-28T12:50:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.933+0000 7f3dbfe00700 1 -- 172.21.15.106:0/1359599560 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0004020 con 0x7f3dc810fe80 2024-04-28T12:50:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.933+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc81a1010 con 0x7f3dc810fe80 2024-04-28T12:50:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.933+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81a88f0 con 0x7f3dc810fe80 2024-04-28T12:50:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.934+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3dc8063bb0 con 0x7f3dc810fe80 2024-04-28T12:50:14.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.936+0000 7f3dbfe00700 1 -- 172.21.15.106:0/1359599560 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3dc001b030 con 0x7f3dc810fe80 2024-04-28T12:50:14.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.938+0000 7f3dbfe00700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3dac0a3cd0 0x7f3dac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:14.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.938+0000 7f3dbfe00700 1 -- 172.21.15.106:0/1359599560 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f3dc00d0d80 con 0x7f3dc810fe80 2024-04-28T12:50:14.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.938+0000 7f3dc7400700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3dac0a3cd0 0x7f3dac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:14.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.938+0000 7f3dbfe00700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] conn(0x7f3dac0aedb0 0x7f3dac0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:14.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.938+0000 7f3dbfe00700 1 -- 172.21.15.106:0/1359599560 --> [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3dac0b1920 con 0x7f3dac0aedb0 2024-04-28T12:50:14.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.938+0000 7f3dbfe00700 1 -- 172.21.15.106:0/1359599560 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f3dc00d1130 con 0x7f3dc810fe80 2024-04-28T12:50:14.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.939+0000 7f3dc6a00700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] conn(0x7f3dac0aedb0 0x7f3dac0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:14.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.939+0000 7f3dc7400700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3dac0a3cd0 0x7f3dac0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006010 tx=0x7f3db800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:14.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.940+0000 7f3dc6a00700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] conn(0x7f3dac0aedb0 0x7f3dac0b1250 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:14.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.941+0000 7f3dbfe00700 1 -- 172.21.15.106:0/1359599560 <== osd.8 v2:172.21.15.171:6816/1721308305 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3dac0b1920 con 0x7f3dac0aedb0 2024-04-28T12:50:14.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.959+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 --> [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3dc8064030 con 0x7f3dac0aedb0 2024-04-28T12:50:14.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.960+0000 7f3dbfe00700 1 -- 172.21.15.106:0/1359599560 <== osd.8 v2:172.21.15.171:6816/1721308305 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3dc8064030 con 0x7f3dac0aedb0 2024-04-28T12:50:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 >> [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] conn(0x7f3dac0aedb0 msgr2=0x7f3dac0b1250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] conn(0x7f3dac0aedb0 0x7f3dac0b1250 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3dac0a3cd0 msgr2=0x7f3dac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:14.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3dac0a3cd0 0x7f3dac0a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006010 tx=0x7f3db800b260 comp rx=0 tx=0).stop 2024-04-28T12:50:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3dc810fe80 msgr2=0x7f3dc81a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 --2- 172.21.15.106:0/1359599560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3dc810fe80 0x7f3dc81a0640 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000ef90 tx=0x7f3dc000c3a0 comp rx=0 tx=0).stop 2024-04-28T12:50:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dc7e00700 1 -- 172.21.15.106:0/1359599560 reap_dead start 2024-04-28T12:50:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 shutdown_connections 2024-04-28T12:50:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 >> 172.21.15.106:0/1359599560 conn(0x7f3dc807a500 msgr2=0x7f3dc8100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 shutdown_connections 2024-04-28T12:50:14.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:14.961+0000 7f3dce800700 1 -- 172.21.15.106:0/1359599560 wait complete. 2024-04-28T12:50:15.198 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:15 smithi106 ceph-mon[28081]: pgmap v360: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:15.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:15 smithi171 ceph-mon[63127]: pgmap v360: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:15.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:15 smithi129 ceph-mon[62418]: pgmap v360: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:15.641 INFO:teuthology.orchestra.run.smithi106.stdout:107374182473 2024-04-28T12:50:15.642 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-28T12:50:15.991 INFO:teuthology.orchestra.run.smithi106.stdout:219043332148 2024-04-28T12:50:15.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-28T12:50:15.998 INFO:teuthology.orchestra.run.smithi106.stdout:287762808870 2024-04-28T12:50:15.998 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-28T12:50:16.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.372+0000 7fabbe400700 1 -- 172.21.15.106:0/1074688218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabb8075f70 msgr2=0x7fabb810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:16.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.372+0000 7fabbe400700 1 --2- 172.21.15.106:0/1074688218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabb8075f70 0x7fabb810f8e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faba0009990 tx=0x7faba0009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:16.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.372+0000 7fabbe400700 1 -- 172.21.15.106:0/1074688218 shutdown_connections 2024-04-28T12:50:16.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.372+0000 7fabbe400700 1 --2- 172.21.15.106:0/1074688218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabb810fe20 0x7fabb8112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:16.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.372+0000 7fabbe400700 1 --2- 172.21.15.106:0/1074688218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb8075650 0x7fabb8075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:16.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.372+0000 7fabbe400700 1 --2- 172.21.15.106:0/1074688218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabb8075f70 0x7fabb810f8e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:16.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.372+0000 7fabbe400700 1 -- 172.21.15.106:0/1074688218 >> 172.21.15.106:0/1074688218 conn(0x7fabb807a580 msgr2=0x7fabb807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:16.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.372+0000 7fabbe400700 1 -- 172.21.15.106:0/1074688218 shutdown_connections 2024-04-28T12:50:16.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.372+0000 7fabbe400700 1 -- 172.21.15.106:0/1074688218 wait complete. 2024-04-28T12:50:16.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.373+0000 7fabbe400700 1 Processor -- start 2024-04-28T12:50:16.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabbe400700 1 -- start start 2024-04-28T12:50:16.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabbe400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabb8075650 0x7fabb819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:16.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabbe400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb8075f70 0x7fabb819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:16.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabbe400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabb810fe20 0x7fabb81a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:16.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabbe400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8115850 con 0x7fabb810fe20 2024-04-28T12:50:16.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabbe400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81156d0 con 0x7fabb8075f70 2024-04-28T12:50:16.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabbe400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81159d0 con 0x7fabb8075650 2024-04-28T12:50:16.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabb7e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabb810fe20 0x7fabb81a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabafe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb8075f70 0x7fabb819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabafe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb8075f70 0x7fabb819c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:47280/0 (socket says 172.21.15.106:47280) 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabafe00700 1 -- 172.21.15.106:0/3575652380 learned_addr learned my addr 172.21.15.106:0/3575652380 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.374+0000 7fabb7400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabb8075650 0x7fabb819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.375+0000 7fabb7e00700 1 -- 172.21.15.106:0/3575652380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabb8075650 msgr2=0x7fabb819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.375+0000 7fabb7e00700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabb8075650 0x7fabb819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.375+0000 7fabb7e00700 1 -- 172.21.15.106:0/3575652380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb8075f70 msgr2=0x7fabb819c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.375+0000 7fabb7e00700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb8075f70 0x7fabb819c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.375+0000 7fabb7e00700 1 -- 172.21.15.106:0/3575652380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb81a0d30 con 0x7fabb810fe20 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.375+0000 7fabafe00700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb8075f70 0x7fabb819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:16.372 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.375+0000 7fabb7e00700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabb810fe20 0x7fabb81a05f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fabb000ef90 tx=0x7fabb000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:16.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.376+0000 7fabb4c00700 1 -- 172.21.15.106:0/3575652380 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb00092e0 con 0x7fabb810fe20 2024-04-28T12:50:16.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.376+0000 7fabb4c00700 1 -- 172.21.15.106:0/3575652380 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb0004bb0 con 0x7fabb810fe20 2024-04-28T12:50:16.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.376+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb81a0fc0 con 0x7fabb810fe20 2024-04-28T12:50:16.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.376+0000 7fabb4c00700 1 -- 172.21.15.106:0/3575652380 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0004020 con 0x7fabb810fe20 2024-04-28T12:50:16.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.376+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb81a88a0 con 0x7fabb810fe20 2024-04-28T12:50:16.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.377+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabb8063bb0 con 0x7fabb810fe20 2024-04-28T12:50:16.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.379+0000 7fabb4c00700 1 -- 172.21.15.106:0/3575652380 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fabb001b030 con 0x7fabb810fe20 2024-04-28T12:50:16.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.380+0000 7fabb4c00700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab9c0a3d50 0x7fab9c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:16.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.380+0000 7fabb4c00700 1 -- 172.21.15.106:0/3575652380 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fabb00d0d90 con 0x7fabb810fe20 2024-04-28T12:50:16.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.380+0000 7fabb4c00700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] conn(0x7fab9c0aee30 0x7fab9c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:16.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.380+0000 7fabb4c00700 1 -- 172.21.15.106:0/3575652380 --> [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab9c0b19a0 con 0x7fab9c0aee30 2024-04-28T12:50:16.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.380+0000 7fabb4c00700 1 -- 172.21.15.106:0/3575652380 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fabb00d1140 con 0x7fabb810fe20 2024-04-28T12:50:16.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.381+0000 7fabb7400700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab9c0a3d50 0x7fab9c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:16.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.381+0000 7fabafe00700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] conn(0x7fab9c0aee30 0x7fab9c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:16.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.381+0000 7fabb7400700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab9c0a3d50 0x7fab9c0a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faba8009c60 tx=0x7faba8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:16.378 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.382+0000 7fabafe00700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] conn(0x7fab9c0aee30 0x7fab9c0b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:16.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.383+0000 7fabb4c00700 1 -- 172.21.15.106:0/3575652380 <== osd.0 v2:172.21.15.106:6802/3844735961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab9c0b19a0 con 0x7fab9c0aee30 2024-04-28T12:50:16.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.401+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 --> [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fabb8063fb0 con 0x7fab9c0aee30 2024-04-28T12:50:16.398 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.402+0000 7fabb4c00700 1 -- 172.21.15.106:0/3575652380 <== osd.0 v2:172.21.15.106:6802/3844735961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fabb8063fb0 con 0x7fab9c0aee30 2024-04-28T12:50:16.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.402+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] conn(0x7fab9c0aee30 msgr2=0x7fab9c0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:16.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.402+0000 7fabbe400700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] conn(0x7fab9c0aee30 0x7fab9c0b12d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:16.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.403+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab9c0a3d50 msgr2=0x7fab9c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:16.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.403+0000 7fabbe400700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fab9c0a3d50 0x7fab9c0a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faba8009c60 tx=0x7faba8009260 comp rx=0 tx=0).stop 2024-04-28T12:50:16.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.403+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabb810fe20 msgr2=0x7fabb81a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:16.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.403+0000 7fabbe400700 1 --2- 172.21.15.106:0/3575652380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabb810fe20 0x7fabb81a05f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fabb000ef90 tx=0x7fabb000c3a0 comp rx=0 tx=0).stop 2024-04-28T12:50:16.399 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.403+0000 7fabb7e00700 1 -- 172.21.15.106:0/3575652380 reap_dead start 2024-04-28T12:50:16.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.403+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 shutdown_connections 2024-04-28T12:50:16.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.403+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 >> 172.21.15.106:0/3575652380 conn(0x7fabb807a580 msgr2=0x7fabb8100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:16.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.403+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 shutdown_connections 2024-04-28T12:50:16.400 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:16.403+0000 7fabbe400700 1 -- 172.21.15.106:0/3575652380 wait complete. 2024-04-28T12:50:16.914 INFO:teuthology.orchestra.run.smithi106.stdout:253403070510 2024-04-28T12:50:16.914 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-28T12:50:17.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:17 smithi171 ceph-mon[63127]: pgmap v361: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:17.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:17 smithi129 ceph-mon[62418]: pgmap v361: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:17.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:17 smithi106 ceph-mon[28081]: pgmap v361: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:17.709 INFO:teuthology.orchestra.run.smithi106.stdout:34359738460 2024-04-28T12:50:17.710 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-28T12:50:17.772 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:17.783 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:17.794 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:17.804 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:17.833 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:17.833 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:17.943 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:17.982 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:18.024 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:19.487 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:19 smithi106 ceph-mon[28081]: pgmap v362: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:50:19.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:19 smithi171 ceph-mon[63127]: pgmap v362: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:50:19.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:19 smithi129 ceph-mon[62418]: pgmap v362: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:50:20.961 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:20.962 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:21.324 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:21 smithi106 ceph-mon[28081]: pgmap v363: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:50:21.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:21 smithi171 ceph-mon[63127]: pgmap v363: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:50:21.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:21 smithi129 ceph-mon[62418]: pgmap v363: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:50:23.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:23 smithi171 ceph-mon[63127]: pgmap v364: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:23.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:23 smithi129 ceph-mon[62418]: pgmap v364: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:23.555 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:23 smithi106 ceph-mon[28081]: pgmap v364: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:25.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:25 smithi171 ceph-mon[63127]: pgmap v365: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:25.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:25 smithi129 ceph-mon[62418]: pgmap v365: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:25.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:25 smithi106 ceph-mon[28081]: pgmap v365: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:26.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:26 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2705441139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-28T12:50:26.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:26 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2705441139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-28T12:50:26.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:26 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2705441139' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-28T12:50:27.445 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:27 smithi106 ceph-mon[28081]: pgmap v366: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:27.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:27 smithi171 ceph-mon[63127]: pgmap v366: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:27.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:27 smithi129 ceph-mon[62418]: pgmap v366: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:28.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.976+0000 7f4ec5c00700 1 -- 172.21.15.106:0/1293723061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ec01024b0 msgr2=0x7f4ec0102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:28.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.976+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/1293723061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ec01024b0 0x7f4ec0102890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009990 tx=0x7f4eb0009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:28.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.976+0000 7f4ec5c00700 1 -- 172.21.15.106:0/1293723061 shutdown_connections 2024-04-28T12:50:28.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.976+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/1293723061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec0107590 0x7f4ec010f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:28.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.976+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/1293723061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ec01024b0 0x7f4ec0102890 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:28.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.976+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/1293723061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec0102e60 0x7f4ec0106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:28.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.976+0000 7f4ec5c00700 1 -- 172.21.15.106:0/1293723061 >> 172.21.15.106:0/1293723061 conn(0x7f4ec00fdd40 msgr2=0x7f4ec0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:28.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.977+0000 7f4ec5c00700 1 -- 172.21.15.106:0/1293723061 shutdown_connections 2024-04-28T12:50:28.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.977+0000 7f4ec5c00700 1 -- 172.21.15.106:0/1293723061 wait complete. 2024-04-28T12:50:28.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.978+0000 7f4ec5c00700 1 Processor -- start 2024-04-28T12:50:28.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.978+0000 7f4ec5c00700 1 -- start start 2024-04-28T12:50:28.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.978+0000 7f4ec5c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec01024b0 0x7f4ec0074e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:28.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ec5c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec0102e60 0x7f4ec006fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:28.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ec5c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ec0107590 0x7f4ec0070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:28.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ec5c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0112470 con 0x7f4ec0102e60 2024-04-28T12:50:28.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ec5c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01122f0 con 0x7f4ec0107590 2024-04-28T12:50:28.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ec5c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec01125f0 con 0x7f4ec01024b0 2024-04-28T12:50:28.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ebfe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec01024b0 0x7f4ec0074e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ebfe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec01024b0 0x7f4ec0074e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49040/0 (socket says 172.21.15.106:49040) 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ebfe00700 1 -- 172.21.15.106:0/3398273279 learned_addr learned my addr 172.21.15.106:0/3398273279 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ec5200700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ec0107590 0x7f4ec0070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ebf400700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec0102e60 0x7f4ec006fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ebfe00700 1 -- 172.21.15.106:0/3398273279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ec0107590 msgr2=0x7f4ec0070310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ebfe00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ec0107590 0x7f4ec0070310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ebfe00700 1 -- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec0102e60 msgr2=0x7f4ec006fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ebfe00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec0102e60 0x7f4ec006fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ebfe00700 1 -- 172.21.15.106:0/3398273279 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec00709f0 con 0x7f4ec01024b0 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.979+0000 7f4ec5200700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ec0107590 0x7f4ec0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.980+0000 7f4ebfe00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec01024b0 0x7f4ec0074e20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005910 tx=0x7f4eb00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:28.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.980+0000 7f4ebcc00700 1 -- 172.21.15.106:0/3398273279 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0027070 con 0x7f4ec01024b0 2024-04-28T12:50:28.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.980+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec006dbc0 con 0x7f4ec01024b0 2024-04-28T12:50:28.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.980+0000 7f4ebcc00700 1 -- 172.21.15.106:0/3398273279 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb0022470 con 0x7f4ec01024b0 2024-04-28T12:50:28.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.980+0000 7f4ebcc00700 1 -- 172.21.15.106:0/3398273279 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0017650 con 0x7f4ec01024b0 2024-04-28T12:50:28.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.980+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec006e0f0 con 0x7f4ec01024b0 2024-04-28T12:50:28.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.982+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec006a110 con 0x7f4ec01024b0 2024-04-28T12:50:28.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.985+0000 7f4ebcc00700 1 -- 172.21.15.106:0/3398273279 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4eb002a070 con 0x7f4ec01024b0 2024-04-28T12:50:28.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.987+0000 7f4ebcc00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4ea40a3cd0 0x7f4ea40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:28.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.987+0000 7f4ebcc00700 1 -- 172.21.15.106:0/3398273279 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f4eb00d16e0 con 0x7f4ec01024b0 2024-04-28T12:50:28.983 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.987+0000 7f4ebf400700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4ea40a3cd0 0x7f4ea40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:28.984 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.988+0000 7f4ebf400700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4ea40a3cd0 0x7f4ea40a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009680 tx=0x7f4eac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:28.987 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.991+0000 7f4ebcc00700 1 -- 172.21.15.106:0/3398273279 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4eb0013070 con 0x7f4ec01024b0 2024-04-28T12:50:29.170 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.099+0000 7f321b000700 1 -- 172.21.15.106:0/611125782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3214103e30 msgr2=0x7f3214110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.099+0000 7f321b000700 1 --2- 172.21.15.106:0/611125782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3214103e30 0x7f3214110290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f320c00b0a0 tx=0x7f320c00b3b0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.099+0000 7f321b000700 1 -- 172.21.15.106:0/611125782 shutdown_connections 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.099+0000 7f321b000700 1 --2- 172.21.15.106:0/611125782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3214103e30 0x7f3214110290 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.099+0000 7f321b000700 1 --2- 172.21.15.106:0/611125782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f32141094c0 0x7f32141098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.099+0000 7f321b000700 1 --2- 172.21.15.106:0/611125782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3214103470 0x7f32141038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.099+0000 7f321b000700 1 -- 172.21.15.106:0/611125782 >> 172.21.15.106:0/611125782 conn(0x7f321407a580 msgr2=0x7f321407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.100+0000 7f321b000700 1 -- 172.21.15.106:0/611125782 shutdown_connections 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.100+0000 7f321b000700 1 -- 172.21.15.106:0/611125782 wait complete. 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.101+0000 7f321b000700 1 Processor -- start 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.101+0000 7f321b000700 1 -- start start 2024-04-28T12:50:29.171 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.101+0000 7f321b000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214103470 0x7f321419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.101+0000 7f321b000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3214103e30 0x7f321419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.101+0000 7f321b000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32141094c0 0x7f32141a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.101+0000 7f321b000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f32141135c0 con 0x7f32141094c0 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.101+0000 7f321b000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3214113440 con 0x7f3214103e30 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.101+0000 7f321b000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3214113740 con 0x7f3214103470 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3219000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32141094c0 0x7f32141a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3213e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214103470 0x7f321419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3219000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32141094c0 0x7f32141a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36708/0 (socket says 172.21.15.106:36708) 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3213e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214103470 0x7f321419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:53322/0 (socket says 172.21.15.106:53322) 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3219000700 1 -- 172.21.15.106:0/2705441139 learned_addr learned my addr 172.21.15.106:0/2705441139 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3213400700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3214103e30 0x7f321419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3219000700 1 -- 172.21.15.106:0/2705441139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214103470 msgr2=0x7f321419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3219000700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214103470 0x7f321419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3219000700 1 -- 172.21.15.106:0/2705441139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3214103e30 msgr2=0x7f321419c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3219000700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3214103e30 0x7f321419c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3219000700 1 -- 172.21.15.106:0/2705441139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32141a0d20 con 0x7f32141094c0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.102+0000 7f3219000700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32141094c0 0x7f32141a05e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f320c00b780 tx=0x7f320c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.103+0000 7f3210c00700 1 -- 172.21.15.106:0/2705441139 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c021070 con 0x7f32141094c0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.103+0000 7f3210c00700 1 -- 172.21.15.106:0/2705441139 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f320c007e30 con 0x7f32141094c0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.103+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32141a0fb0 con 0x7f32141094c0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.103+0000 7f3210c00700 1 -- 172.21.15.106:0/2705441139 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c004700 con 0x7f32141094c0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.103+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32141a8890 con 0x7f32141094c0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.104+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321406a110 con 0x7f32141094c0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.106+0000 7f3210c00700 1 -- 172.21.15.106:0/2705441139 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f320c007980 con 0x7f32141094c0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.107+0000 7f3210c00700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31f80a3d50 0x7f31f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.107+0000 7f3210c00700 1 -- 172.21.15.106:0/2705441139 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f320c00f020 con 0x7f32141094c0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.108+0000 7f3213e00700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31f80a3d50 0x7f31f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.108+0000 7f3213e00700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31f80a3d50 0x7f31f80a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3204009be0 tx=0x7f3204009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.112+0000 7f3210c00700 1 -- 172.21.15.106:0/2705441139 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f320c02a020 con 0x7f32141094c0 2024-04-28T12:50:29.174 INFO:teuthology.orchestra.run.smithi106.stdout:197568495676 2024-04-28T12:50:29.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.358+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f32141a8a60 con 0x7f32141094c0 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.358+0000 7f3210c00700 1 -- 172.21.15.106:0/2705441139 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f320c09e120 con 0x7f32141094c0 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31f80a3d50 msgr2=0x7f31f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31f80a3d50 0x7f31f80a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3204009be0 tx=0x7f3204009260 comp rx=0 tx=0).stop 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32141094c0 msgr2=0x7f32141a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32141094c0 0x7f32141a05e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f320c00b780 tx=0x7f320c004d00 comp rx=0 tx=0).stop 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 shutdown_connections 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32141094c0 0x7f32141a05e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31f80a3d50 0x7f31f80a6210 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3214103470 0x7f321419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 --2- 172.21.15.106:0/2705441139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3214103e30 0x7f321419c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 >> 172.21.15.106:0/2705441139 conn(0x7f321407a580 msgr2=0x7f321410e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 shutdown_connections 2024-04-28T12:50:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:25.362+0000 7f321b000700 1 -- 172.21.15.106:0/2705441139 wait complete. 2024-04-28T12:50:29.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.225+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4ec0070bc0 con 0x7f4ec01024b0 2024-04-28T12:50:29.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.226+0000 7f4ebcc00700 1 -- 172.21.15.106:0/3398273279 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4eb009dc50 con 0x7f4ec01024b0 2024-04-28T12:50:29.222 INFO:teuthology.orchestra.run.smithi106.stdout:141733920840 2024-04-28T12:50:29.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.228+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4ea40a3cd0 msgr2=0x7f4ea40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:29.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.228+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4ea40a3cd0 0x7f4ea40a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009680 tx=0x7f4eac009260 comp rx=0 tx=0).stop 2024-04-28T12:50:29.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.228+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec01024b0 msgr2=0x7f4ec0074e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:29.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.228+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec01024b0 0x7f4ec0074e20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0005910 tx=0x7f4eb00047b0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.229+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 shutdown_connections 2024-04-28T12:50:29.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.229+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ec0107590 0x7f4ec0070310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.229+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4ea40a3cd0 0x7f4ea40a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.229+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ec01024b0 0x7f4ec0074e20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.229+0000 7f4ec5c00700 1 --2- 172.21.15.106:0/3398273279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ec0102e60 0x7f4ec006fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:29.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.229+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 >> 172.21.15.106:0/3398273279 conn(0x7f4ec00fdd40 msgr2=0x7f4ec00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:29.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.229+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 shutdown_connections 2024-04-28T12:50:29.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.229+0000 7f4ec5c00700 1 -- 172.21.15.106:0/3398273279 wait complete. 2024-04-28T12:50:29.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:29 smithi171 ceph-mon[63127]: pgmap v367: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:50:29.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:29 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1811397101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-28T12:50:29.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:29 smithi129 ceph-mon[62418]: pgmap v367: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:50:29.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:29 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1811397101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-28T12:50:29.542 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:29 smithi106 ceph-mon[28081]: pgmap v367: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:50:29.543 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:29 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1811397101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-28T12:50:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.901+0000 7f1851800700 1 -- 172.21.15.106:0/4122684990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c1094c0 msgr2=0x7f184c1098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.901+0000 7f1851800700 1 --2- 172.21.15.106:0/4122684990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c1094c0 0x7f184c1098a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f183c009990 tx=0x7f183c009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.254 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.901+0000 7f1851800700 1 -- 172.21.15.106:0/4122684990 shutdown_connections 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.901+0000 7f1851800700 1 --2- 172.21.15.106:0/4122684990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f184c103e30 0x7f184c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.901+0000 7f1851800700 1 --2- 172.21.15.106:0/4122684990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c1094c0 0x7f184c1098a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.901+0000 7f1851800700 1 --2- 172.21.15.106:0/4122684990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f184c103470 0x7f184c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.901+0000 7f1851800700 1 -- 172.21.15.106:0/4122684990 >> 172.21.15.106:0/4122684990 conn(0x7f184c07a580 msgr2=0x7f184c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.901+0000 7f1851800700 1 -- 172.21.15.106:0/4122684990 shutdown_connections 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.901+0000 7f1851800700 1 -- 172.21.15.106:0/4122684990 wait complete. 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.902+0000 7f1851800700 1 Processor -- start 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f1851800700 1 -- start start 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f1851800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c103470 0x7f184c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f1851800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f184c103e30 0x7f184c075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f1851800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f184c1094c0 0x7f184c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f1851800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f184c113690 con 0x7f184c103470 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f1851800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f184c113510 con 0x7f184c103e30 2024-04-28T12:50:30.255 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f1851800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f184c113810 con 0x7f184c1094c0 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f184aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c103470 0x7f184c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f184aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c103470 0x7f184c075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35872/0 (socket says 172.21.15.106:35872) 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f184aa00700 1 -- 172.21.15.106:0/1811397101 learned_addr learned my addr 172.21.15.106:0/1811397101 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.903+0000 7f184a000700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f184c103e30 0x7f184c075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.904+0000 7f184b400700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f184c1094c0 0x7f184c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.904+0000 7f184aa00700 1 -- 172.21.15.106:0/1811397101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f184c1094c0 msgr2=0x7f184c0761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.904+0000 7f184aa00700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f184c1094c0 0x7f184c0761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.904+0000 7f184aa00700 1 -- 172.21.15.106:0/1811397101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f184c103e30 msgr2=0x7f184c075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.904+0000 7f184aa00700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f184c103e30 0x7f184c075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.904+0000 7f184aa00700 1 -- 172.21.15.106:0/1811397101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c1a80f0 con 0x7f184c103470 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.904+0000 7f184aa00700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c103470 0x7f184c075720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f183c000c00 tx=0x7f183c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.905+0000 7f1843400700 1 -- 172.21.15.106:0/1811397101 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c027070 con 0x7f184c103470 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.905+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c1a8380 con 0x7f184c103470 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.905+0000 7f1843400700 1 -- 172.21.15.106:0/1811397101 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183c022470 con 0x7f184c103470 2024-04-28T12:50:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.905+0000 7f1843400700 1 -- 172.21.15.106:0/1811397101 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c017650 con 0x7f184c103470 2024-04-28T12:50:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.905+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1a88b0 con 0x7f184c103470 2024-04-28T12:50:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.906+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c06a110 con 0x7f184c103470 2024-04-28T12:50:30.257 INFO:teuthology.orchestra.run.smithi106.stdout:253403070513 2024-04-28T12:50:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.909+0000 7f1843400700 1 -- 172.21.15.106:0/1811397101 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f183c02a070 con 0x7f184c103470 2024-04-28T12:50:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.910+0000 7f1843400700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18300a3cd0 0x7f18300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.911+0000 7f1843400700 1 -- 172.21.15.106:0/1811397101 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f183c0d19e0 con 0x7f184c103470 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.911+0000 7f184a000700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18300a3cd0 0x7f18300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.911+0000 7f184a000700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18300a3cd0 0x7f18300a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f184c07ac60 tx=0x7f1838009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:28.913+0000 7f1843400700 1 -- 172.21.15.106:0/1811397101 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f183c013070 con 0x7f184c103470 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.149+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f184c076c50 con 0x7f184c103470 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.149+0000 7f1843400700 1 -- 172.21.15.106:0/1811397101 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f183c09df50 con 0x7f184c103470 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18300a3cd0 msgr2=0x7f18300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18300a3cd0 0x7f18300a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f184c07ac60 tx=0x7f1838009260 comp rx=0 tx=0).stop 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c103470 msgr2=0x7f184c075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c103470 0x7f184c075720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f183c000c00 tx=0x7f183c0057c0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 shutdown_connections 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f18300a3cd0 0x7f18300a6190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f184c1094c0 0x7f184c0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f184c103470 0x7f184c075720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 --2- 172.21.15.106:0/1811397101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f184c103e30 0x7f184c075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 >> 172.21.15.106:0/1811397101 conn(0x7f184c07a580 msgr2=0x7f184c102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 shutdown_connections 2024-04-28T12:50:30.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:29.153+0000 7f1851800700 1 -- 172.21.15.106:0/1811397101 wait complete. 2024-04-28T12:50:30.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:30 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3398273279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-28T12:50:30.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:30 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3398273279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-28T12:50:30.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:30 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3398273279' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-28T12:50:31.129 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.132+0000 7f1de9200700 1 -- 172.21.15.106:0/1961381312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de410b660 msgr2=0x7f1de410ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.132+0000 7f1de9200700 1 --2- 172.21.15.106:0/1961381312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de410b660 0x7f1de410ba40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009990 tx=0x7f1dd4009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.133+0000 7f1de9200700 1 -- 172.21.15.106:0/1961381312 shutdown_connections 2024-04-28T12:50:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.133+0000 7f1de9200700 1 --2- 172.21.15.106:0/1961381312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de4106020 0x7f1de4112530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.133+0000 7f1de9200700 1 --2- 172.21.15.106:0/1961381312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de410b660 0x7f1de410ba40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.133+0000 7f1de9200700 1 --2- 172.21.15.106:0/1961381312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de4105660 0x7f1de4105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.133+0000 7f1de9200700 1 -- 172.21.15.106:0/1961381312 >> 172.21.15.106:0/1961381312 conn(0x7f1de4101150 msgr2=0x7f1de4103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.133+0000 7f1de9200700 1 -- 172.21.15.106:0/1961381312 shutdown_connections 2024-04-28T12:50:31.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.133+0000 7f1de9200700 1 -- 172.21.15.106:0/1961381312 wait complete. 2024-04-28T12:50:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.134+0000 7f1de9200700 1 Processor -- start 2024-04-28T12:50:31.131 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.135+0000 7f1de9200700 1 -- start start 2024-04-28T12:50:31.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.135+0000 7f1de9200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de4105660 0x7f1de41a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:31.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de9200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de4106020 0x7f1de41a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:31.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de9200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de410b660 0x7f1de419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:31.132 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de9200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1de4115840 con 0x7f1de4106020 2024-04-28T12:50:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de9200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41156c0 con 0x7f1de4105660 2024-04-28T12:50:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de9200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1de41159c0 con 0x7f1de410b660 2024-04-28T12:50:31.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de1e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de4105660 0x7f1de41a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de1e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de4105660 0x7f1de41a1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:59980/0 (socket says 172.21.15.106:59980) 2024-04-28T12:50:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de1e00700 1 -- 172.21.15.106:0/1385178011 learned_addr learned my addr 172.21.15.106:0/1385178011 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1ddbe00700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de4106020 0x7f1de41a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de2800700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de410b660 0x7f1de419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de1e00700 1 -- 172.21.15.106:0/1385178011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de410b660 msgr2=0x7f1de419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de1e00700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de410b660 0x7f1de419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de1e00700 1 -- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de4106020 msgr2=0x7f1de41a1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:31.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de1e00700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de4106020 0x7f1de41a1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.136+0000 7f1de1e00700 1 -- 172.21.15.106:0/1385178011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de419c2a0 con 0x7f1de4105660 2024-04-28T12:50:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.137+0000 7f1de1e00700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de4105660 0x7f1de41a1930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40057c0 tx=0x7f1dd40057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.137+0000 7f1ddaa00700 1 -- 172.21.15.106:0/1385178011 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4027070 con 0x7f1de4105660 2024-04-28T12:50:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.137+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de419c530 con 0x7f1de4105660 2024-04-28T12:50:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.137+0000 7f1ddaa00700 1 -- 172.21.15.106:0/1385178011 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd4003f20 con 0x7f1de4105660 2024-04-28T12:50:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.137+0000 7f1ddaa00700 1 -- 172.21.15.106:0/1385178011 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4022b50 con 0x7f1de4105660 2024-04-28T12:50:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.137+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de41a8800 con 0x7f1de4105660 2024-04-28T12:50:31.135 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.139+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de410dbf0 con 0x7f1de4105660 2024-04-28T12:50:31.138 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.141+0000 7f1ddaa00700 1 -- 172.21.15.106:0/1385178011 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1dd402a070 con 0x7f1de4105660 2024-04-28T12:50:31.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.142+0000 7f1ddaa00700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:31.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.142+0000 7f1ddaa00700 1 -- 172.21.15.106:0/1385178011 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f1dd40d1c10 con 0x7f1de4105660 2024-04-28T12:50:31.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.142+0000 7f1ddbe00700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:31.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.143+0000 7f1ddbe00700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1de4103810 tx=0x7f1dcc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:31.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.144+0000 7f1ddaa00700 1 -- 172.21.15.106:0/1385178011 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1dd4013070 con 0x7f1de4105660 2024-04-28T12:50:31.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.376+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f1de419d010 con 0x7f1de4105660 2024-04-28T12:50:31.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.377+0000 7f1ddaa00700 1 -- 172.21.15.106:0/1385178011 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f1de419d010 con 0x7f1de4105660 2024-04-28T12:50:31.373 INFO:teuthology.orchestra.run.smithi106.stdout:309237645349 2024-04-28T12:50:31.376 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dc80a3cd0 msgr2=0x7f1dc80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1de4103810 tx=0x7f1dcc00b410 comp rx=0 tx=0).stop 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de4105660 msgr2=0x7f1de41a1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de4105660 0x7f1de41a1930 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1dd40057c0 tx=0x7f1dd40057f0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 shutdown_connections 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dc80a3cd0 0x7f1dc80a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1de410b660 0x7f1de419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1de4105660 0x7f1de41a1930 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 --2- 172.21.15.106:0/1385178011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de4106020 0x7f1de41a1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 >> 172.21.15.106:0/1385178011 conn(0x7f1de4101150 msgr2=0x7f1de4101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 shutdown_connections 2024-04-28T12:50:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:31.380+0000 7f1de9200700 1 -- 172.21.15.106:0/1385178011 wait complete. 2024-04-28T12:50:31.475 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:31 smithi106 ceph-mon[28081]: pgmap v368: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:50:31.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:31 smithi171 ceph-mon[63127]: pgmap v368: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:50:31.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:31 smithi129 ceph-mon[62418]: pgmap v368: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:50:32.421 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:32 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1385178011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-28T12:50:32.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:32 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1385178011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-28T12:50:32.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:32 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1385178011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-28T12:50:32.560 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495673 got 197568495676 for osd.6 2024-04-28T12:50:32.560 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.424+0000 7fdda3800700 1 -- 172.21.15.106:0/4291701514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd9c1094c0 msgr2=0x7fdd9c1098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.424+0000 7fdda3800700 1 --2- 172.21.15.106:0/4291701514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd9c1094c0 0x7fdd9c1098a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdd900098f0 tx=0x7fdd90009c00 comp rx=0 tx=0).stop 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.425+0000 7fdda3800700 1 -- 172.21.15.106:0/4291701514 shutdown_connections 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.425+0000 7fdda3800700 1 --2- 172.21.15.106:0/4291701514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd9c103e30 0x7fdd9c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.425+0000 7fdda3800700 1 --2- 172.21.15.106:0/4291701514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd9c1094c0 0x7fdd9c1098a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.425+0000 7fdda3800700 1 --2- 172.21.15.106:0/4291701514 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd9c103470 0x7fdd9c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.425+0000 7fdda3800700 1 -- 172.21.15.106:0/4291701514 >> 172.21.15.106:0/4291701514 conn(0x7fdd9c07a580 msgr2=0x7fdd9c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.425+0000 7fdda3800700 1 -- 172.21.15.106:0/4291701514 shutdown_connections 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.425+0000 7fdda3800700 1 -- 172.21.15.106:0/4291701514 wait complete. 2024-04-28T12:50:32.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.426+0000 7fdda3800700 1 Processor -- start 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.426+0000 7fdda3800700 1 -- start start 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.426+0000 7fdda3800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd9c103470 0x7fdd9c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.426+0000 7fdda3800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.426+0000 7fdda3800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd9c1094c0 0x7fdd9c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.426+0000 7fdda3800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113610 con 0x7fdd9c1094c0 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.426+0000 7fdda3800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113490 con 0x7fdd9c103470 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.426+0000 7fdda3800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113790 con 0x7fdd9c103e30 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdd9be00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdd9be00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49080/0 (socket says 172.21.15.106:49080) 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdd9be00700 1 -- 172.21.15.106:0/2113265523 learned_addr learned my addr 172.21.15.106:0/2113265523 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdda1800700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd9c1094c0 0x7fdd9c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdda0e00700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd9c103470 0x7fdd9c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdd9be00700 1 -- 172.21.15.106:0/2113265523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd9c103470 msgr2=0x7fdd9c19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:32.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdd9be00700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd9c103470 0x7fdd9c19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdd9be00700 1 -- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd9c1094c0 msgr2=0x7fdd9c1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdd9be00700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd9c1094c0 0x7fdd9c1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.427+0000 7fdd9be00700 1 -- 172.21.15.106:0/2113265523 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c1a0e70 con 0x7fdd9c103e30 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.428+0000 7fdd9be00700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19c3a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800e9c0 tx=0x7fdd8800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.428+0000 7fdd99600700 1 -- 172.21.15.106:0/2113265523 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88004a30 con 0x7fdd9c103e30 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.428+0000 7fdd99600700 1 -- 172.21.15.106:0/2113265523 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd8800cc70 con 0x7fdd9c103e30 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.428+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c1a1160 con 0x7fdd9c103e30 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.428+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c1a89e0 con 0x7fdd9c103e30 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.428+0000 7fdd99600700 1 -- 172.21.15.106:0/2113265523 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88005490 con 0x7fdd9c103e30 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.429+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd7c005330 con 0x7fdd9c103e30 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.431+0000 7fdd99600700 1 -- 172.21.15.106:0/2113265523 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdd8800c7c0 con 0x7fdd9c103e30 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.432+0000 7fdd99600700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdd840a3d50 0x7fdd840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.432+0000 7fdd99600700 1 -- 172.21.15.106:0/2113265523 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7fdd880d0870 con 0x7fdd9c103e30 2024-04-28T12:50:32.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.433+0000 7fdda0e00700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdd840a3d50 0x7fdd840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:32.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.433+0000 7fdda0e00700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdd840a3d50 0x7fdd840a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdd900098c0 tx=0x7fdd90019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:32.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.437+0000 7fdd99600700 1 -- 172.21.15.106:0/2113265523 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fdd8800ce20 con 0x7fdd9c103e30 2024-04-28T12:50:32.912 INFO:teuthology.orchestra.run.smithi106.stdout:287762808875 2024-04-28T12:50:32.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.672+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fdd7c0058f0 con 0x7fdd9c103e30 2024-04-28T12:50:32.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.672+0000 7fdd99600700 1 -- 172.21.15.106:0/2113265523 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdd8809cd50 con 0x7fdd9c103e30 2024-04-28T12:50:32.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.675+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdd840a3d50 msgr2=0x7fdd840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:32.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.675+0000 7fdda3800700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdd840a3d50 0x7fdd840a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdd900098c0 tx=0x7fdd90019040 comp rx=0 tx=0).stop 2024-04-28T12:50:32.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.675+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd9c103e30 msgr2=0x7fdd9c19c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:32.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.675+0000 7fdda3800700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19c3a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800e9c0 tx=0x7fdd8800ecd0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.676+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 shutdown_connections 2024-04-28T12:50:32.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.676+0000 7fdda3800700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdd9c1094c0 0x7fdd9c1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.676+0000 7fdda3800700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd9c103470 0x7fdd9c19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.676+0000 7fdda3800700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fdd840a3d50 0x7fdd840a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.676+0000 7fdda3800700 1 --2- 172.21.15.106:0/2113265523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd9c103e30 0x7fdd9c19c3a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:32.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.676+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 >> 172.21.15.106:0/2113265523 conn(0x7fdd9c07a580 msgr2=0x7fdd9c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:32.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.676+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 shutdown_connections 2024-04-28T12:50:32.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:32.676+0000 7fdda3800700 1 -- 172.21.15.106:0/2113265523 wait complete. 2024-04-28T12:50:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:33 smithi171 ceph-mon[63127]: pgmap v369: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:33 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2113265523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-28T12:50:33.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:33 smithi129 ceph-mon[62418]: pgmap v369: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:33 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2113265523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-28T12:50:33.546 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:33 smithi106 ceph-mon[28081]: pgmap v369: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:33.546 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:33 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2113265523' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-28T12:50:33.964 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808870 got 287762808875 for osd.9 2024-04-28T12:50:33.964 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:34.075 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920836 got 141733920840 for osd.4 2024-04-28T12:50:34.075 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:34.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:34 smithi106 ceph-mon[28081]: pgmap v370: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:34.745 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:34 smithi171 ceph-mon[63127]: pgmap v370: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:34.763 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:34 smithi129 ceph-mon[62418]: pgmap v370: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:35.933 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070510 got 253403070513 for osd.8 2024-04-28T12:50:35.933 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:35.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.888+0000 7f0b1cc00700 1 -- 172.21.15.106:0/3136449885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b18108380 msgr2=0x7f0b18108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:35.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.888+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/3136449885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b18108380 0x7f0b18108760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009990 tx=0x7f0b08009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:35.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.888+0000 7f0b1cc00700 1 -- 172.21.15.106:0/3136449885 shutdown_connections 2024-04-28T12:50:35.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.888+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/3136449885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b18102cd0 0x7f0b1810f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.888+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/3136449885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b18108380 0x7f0b18108760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.888+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/3136449885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b18102310 0x7f0b18102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.888+0000 7f0b1cc00700 1 -- 172.21.15.106:0/3136449885 >> 172.21.15.106:0/3136449885 conn(0x7f0b18079290 msgr2=0x7f0b180796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.889+0000 7f0b1cc00700 1 -- 172.21.15.106:0/3136449885 shutdown_connections 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.889+0000 7f0b1cc00700 1 -- 172.21.15.106:0/3136449885 wait complete. 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.891+0000 7f0b1cc00700 1 Processor -- start 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.891+0000 7f0b1cc00700 1 -- start start 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.891+0000 7f0b1cc00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b18102310 0x7f0b1819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.891+0000 7f0b1cc00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b18102cd0 0x7f0b1819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.891+0000 7f0b1cc00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b18108380 0x7f0b181a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.891+0000 7f0b1cc00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b18112520 con 0x7f0b18102310 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.891+0000 7f0b1cc00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0b181123a0 con 0x7f0b18102cd0 2024-04-28T12:50:35.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.891+0000 7f0b1cc00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0b181126a0 con 0x7f0b18108380 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b15e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b18102310 0x7f0b1819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b15400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b18102cd0 0x7f0b1819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b15e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b18102310 0x7f0b1819bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35922/0 (socket says 172.21.15.106:35922) 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b15e00700 1 -- 172.21.15.106:0/1320192764 learned_addr learned my addr 172.21.15.106:0/1320192764 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b16800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b18108380 0x7f0b181a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b15400700 1 -- 172.21.15.106:0/1320192764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b18108380 msgr2=0x7f0b181a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b15400700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b18108380 0x7f0b181a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b15400700 1 -- 172.21.15.106:0/1320192764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b18102310 msgr2=0x7f0b1819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b15400700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b18102310 0x7f0b1819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b15400700 1 -- 172.21.15.106:0/1320192764 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b181a0e00 con 0x7f0b18102cd0 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.892+0000 7f0b16800700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b18108380 0x7f0b181a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.893+0000 7f0b15400700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b18102cd0 0x7f0b1819c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400e9f0 tx=0x7f0b0400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.893+0000 7f0b0ea00700 1 -- 172.21.15.106:0/1320192764 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04004220 con 0x7f0b18102cd0 2024-04-28T12:50:35.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.893+0000 7f0b0ea00700 1 -- 172.21.15.106:0/1320192764 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b040043c0 con 0x7f0b18102cd0 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.893+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b181a10f0 con 0x7f0b18102cd0 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.893+0000 7f0b0ea00700 1 -- 172.21.15.106:0/1320192764 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04005020 con 0x7f0b18102cd0 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.893+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b181a8970 con 0x7f0b18102cd0 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.894+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1806a110 con 0x7f0b18102cd0 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.897+0000 7f0b0ea00700 1 -- 172.21.15.106:0/1320192764 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0b04004750 con 0x7f0b18102cd0 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.898+0000 7f0b0ea00700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0afc0a3d50 0x7f0afc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.898+0000 7f0b0ea00700 1 -- 172.21.15.106:0/1320192764 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f0b040d08d0 con 0x7f0b18102cd0 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.899+0000 7f0b15e00700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0afc0a3d50 0x7f0afc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.899+0000 7f0b15e00700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0afc0a3d50 0x7f0afc0a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006010 tx=0x7f0b0800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:35.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:35.900+0000 7f0b0ea00700 1 -- 172.21.15.106:0/1320192764 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0b0409ce40 con 0x7f0b18102cd0 2024-04-28T12:50:36.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.132+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f0b181a8ba0 con 0x7f0b18102cd0 2024-04-28T12:50:36.130 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.134+0000 7f0b0ea00700 1 -- 172.21.15.106:0/1320192764 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0b0409c740 con 0x7f0b18102cd0 2024-04-28T12:50:36.130 INFO:teuthology.orchestra.run.smithi106.stdout:219043332153 2024-04-28T12:50:36.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.136+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0afc0a3d50 msgr2=0x7f0afc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:36.133 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.136+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0afc0a3d50 0x7f0afc0a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b08006010 tx=0x7f0b0800b280 comp rx=0 tx=0).stop 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.136+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b18102cd0 msgr2=0x7f0b1819c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.136+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b18102cd0 0x7f0b1819c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400e9f0 tx=0x7f0b0400ed00 comp rx=0 tx=0).stop 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.137+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 shutdown_connections 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.137+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0afc0a3d50 0x7f0afc0a6210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.137+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0b18108380 0x7f0b181a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.137+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b18102310 0x7f0b1819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.137+0000 7f0b1cc00700 1 --2- 172.21.15.106:0/1320192764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0b18102cd0 0x7f0b1819c510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.137+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 >> 172.21.15.106:0/1320192764 conn(0x7f0b18079290 msgr2=0x7f0b18100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.137+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 shutdown_connections 2024-04-28T12:50:36.134 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:36.137+0000 7f0b1cc00700 1 -- 172.21.15.106:0/1320192764 wait complete. 2024-04-28T12:50:36.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:36 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1320192764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-28T12:50:36.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:36 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1320192764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-28T12:50:36.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:36 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1320192764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-28T12:50:36.937 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645349 for osd.10 2024-04-28T12:50:36.937 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:37.335 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:37 smithi106 ceph-mon[28081]: pgmap v371: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:37.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:37 smithi171 ceph-mon[63127]: pgmap v371: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:37.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:37 smithi129 ceph-mon[62418]: pgmap v371: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:38.316 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:38 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2687311693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-28T12:50:38.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:38 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2687311693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-28T12:50:38.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:38 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2687311693' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.362+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2949949100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105f10 msgr2=0x7ff4c41123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.362+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2949949100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105f10 0x7ff4c41123d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00b0a0 tx=0x7ff4bc00b3b0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.362+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2949949100 shutdown_connections 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.362+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2949949100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105f10 0x7ff4c41123d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.362+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2949949100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4c410b600 0x7ff4c410b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.362+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2949949100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4c4105550 0x7ff4c41059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.362+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2949949100 >> 172.21.15.106:0/2949949100 conn(0x7ff4c4101060 msgr2=0x7ff4c4103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.363+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2949949100 shutdown_connections 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.363+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2949949100 wait complete. 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.364+0000 7ff4c9e00700 1 Processor -- start 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.364+0000 7ff4c9e00700 1 -- start start 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.364+0000 7ff4c9e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105550 0x7ff4c41a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:38.517 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.364+0000 7ff4c9e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4c4105f10 0x7ff4c41a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.364+0000 7ff4c9e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4c410b600 0x7ff4c419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.364+0000 7ff4c9e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c4115760 con 0x7ff4c410b600 2024-04-28T12:50:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.364+0000 7ff4c9e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41155e0 con 0x7ff4c4105f10 2024-04-28T12:50:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.364+0000 7ff4c9e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c41158e0 con 0x7ff4c4105550 2024-04-28T12:50:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c8a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105550 0x7ff4c41a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c8a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105550 0x7ff4c41a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49110/0 (socket says 172.21.15.106:49110) 2024-04-28T12:50:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c8a00700 1 -- 172.21.15.106:0/2687311693 learned_addr learned my addr 172.21.15.106:0/2687311693 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:38.518 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c9400700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4c410b600 0x7ff4c419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:38.518 INFO:teuthology.orchestra.run.smithi106.stdout:77309411414 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c3e00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4c4105f10 0x7ff4c41a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c9400700 1 -- 172.21.15.106:0/2687311693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105550 msgr2=0x7ff4c41a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c9400700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105550 0x7ff4c41a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c9400700 1 -- 172.21.15.106:0/2687311693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4c4105f10 msgr2=0x7ff4c41a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c9400700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4c4105f10 0x7ff4c41a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c9400700 1 -- 172.21.15.106:0/2687311693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c419c0c0 con 0x7ff4c410b600 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.365+0000 7ff4c8a00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105550 0x7ff4c41a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.366+0000 7ff4c9400700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4c410b600 0x7ff4c419b960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00b070 tx=0x7ff4bc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.366+0000 7ff4c1600700 1 -- 172.21.15.106:0/2687311693 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc021070 con 0x7ff4c410b600 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.366+0000 7ff4c1600700 1 -- 172.21.15.106:0/2687311693 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4bc007b80 con 0x7ff4c410b600 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.366+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c419c350 con 0x7ff4c410b600 2024-04-28T12:50:38.519 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.366+0000 7ff4c1600700 1 -- 172.21.15.106:0/2687311693 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc005040 con 0x7ff4c410b600 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.366+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c41a8620 con 0x7ff4c410b600 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.367+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c406a110 con 0x7ff4c410b600 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.369+0000 7ff4c1600700 1 -- 172.21.15.106:0/2687311693 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff4bc024070 con 0x7ff4c410b600 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.370+0000 7ff4c1600700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff4ac0a3cd0 0x7ff4ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.371+0000 7ff4c1600700 1 -- 172.21.15.106:0/2687311693 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7ff4bc0d1760 con 0x7ff4c410b600 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.371+0000 7ff4c8a00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff4ac0a3cd0 0x7ff4ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.371+0000 7ff4c8a00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff4ac0a3cd0 0x7ff4ac0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff4c410a610 tx=0x7ff4b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.375+0000 7ff4c1600700 1 -- 172.21.15.106:0/2687311693 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff4bc09dcd0 con 0x7ff4c410b600 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.605+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff4c402e490 con 0x7ff4c410b600 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.606+0000 7ff4c1600700 1 -- 172.21.15.106:0/2687311693 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff4bc09d5d0 con 0x7ff4c410b600 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff4ac0a3cd0 msgr2=0x7ff4ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff4ac0a3cd0 0x7ff4ac0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff4c410a610 tx=0x7ff4b8009260 comp rx=0 tx=0).stop 2024-04-28T12:50:38.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4c410b600 msgr2=0x7ff4c419b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4c410b600 0x7ff4c419b960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00b070 tx=0x7ff4bc004c80 comp rx=0 tx=0).stop 2024-04-28T12:50:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 shutdown_connections 2024-04-28T12:50:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4c410b600 0x7ff4c419b960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff4ac0a3cd0 0x7ff4ac0a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff4c4105550 0x7ff4c41a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 --2- 172.21.15.106:0/2687311693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff4c4105f10 0x7ff4c41a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.609+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 >> 172.21.15.106:0/2687311693 conn(0x7ff4c4101060 msgr2=0x7ff4c4102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.610+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 shutdown_connections 2024-04-28T12:50:38.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:37.610+0000 7ff4c9e00700 1 -- 172.21.15.106:0/2687311693 wait complete. 2024-04-28T12:50:39.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.107+0000 7f01a0200700 1 -- 172.21.15.106:0/1213376177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 msgr2=0x7f0198075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.107+0000 7f01a0200700 1 --2- 172.21.15.106:0/1213376177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 0x7f0198075a30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f018c009990 tx=0x7f018c009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.107+0000 7f01a0200700 1 -- 172.21.15.106:0/1213376177 shutdown_connections 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.107+0000 7f01a0200700 1 --2- 172.21.15.106:0/1213376177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f019810fe20 0x7f0198112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.107+0000 7f01a0200700 1 --2- 172.21.15.106:0/1213376177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 0x7f0198075a30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.107+0000 7f01a0200700 1 --2- 172.21.15.106:0/1213376177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0198075f70 0x7f019810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.107+0000 7f01a0200700 1 -- 172.21.15.106:0/1213376177 >> 172.21.15.106:0/1213376177 conn(0x7f019807a580 msgr2=0x7f019807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.107+0000 7f01a0200700 1 -- 172.21.15.106:0/1213376177 shutdown_connections 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.107+0000 7f01a0200700 1 -- 172.21.15.106:0/1213376177 wait complete. 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.108+0000 7f01a0200700 1 Processor -- start 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f01a0200700 1 -- start start 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f01a0200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 0x7f01981a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:39.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f01a0200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0198075f70 0x7f01981a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f01a0200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f019810fe20 0x7f019819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f01a0200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0198115820 con 0x7f0198075650 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f01a0200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f01981156a0 con 0x7f019810fe20 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f01a0200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f01981159a0 con 0x7f0198075f70 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f019d800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 0x7f01981a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f019d800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 0x7f01981a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35258/0 (socket says 172.21.15.106:35258) 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.109+0000 7f019d800700 1 -- 172.21.15.106:0/3457039463 learned_addr learned my addr 172.21.15.106:0/3457039463 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.110+0000 7f019e200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f019810fe20 0x7f019819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.110+0000 7f019ce00700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0198075f70 0x7f01981a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.110+0000 7f019e200700 1 -- 172.21.15.106:0/3457039463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0198075f70 msgr2=0x7f01981a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.110+0000 7f019e200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0198075f70 0x7f01981a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.110+0000 7f019e200700 1 -- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 msgr2=0x7f01981a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.110+0000 7f019e200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 0x7f01981a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.110+0000 7f019e200700 1 -- 172.21.15.106:0/3457039463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019819c400 con 0x7f019810fe20 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.110+0000 7f019d800700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 0x7f01981a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.110+0000 7f019e200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f019810fe20 0x7f019819bb20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f019000ef90 tx=0x7f019000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.111+0000 7f0196000700 1 -- 172.21.15.106:0/3457039463 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01900092e0 con 0x7f019810fe20 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.111+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019819c690 con 0x7f019810fe20 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.111+0000 7f0196000700 1 -- 172.21.15.106:0/3457039463 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0190004bb0 con 0x7f019810fe20 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.111+0000 7f0196000700 1 -- 172.21.15.106:0/3457039463 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190004020 con 0x7f019810fe20 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.111+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01981a89d0 con 0x7f019810fe20 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.112+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019819ca00 con 0x7f019810fe20 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.114+0000 7f0196000700 1 -- 172.21.15.106:0/3457039463 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f019001b030 con 0x7f019810fe20 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.115+0000 7f0196000700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f01800a3cd0 0x7f01800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.115+0000 7f0196000700 1 -- 172.21.15.106:0/3457039463 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f01900d0e30 con 0x7f019810fe20 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.116+0000 7f019d800700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f01800a3cd0 0x7f01800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.116+0000 7f019d800700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f01800a3cd0 0x7f01800a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f018c006010 tx=0x7f018c00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:39.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.118+0000 7f0196000700 1 -- 172.21.15.106:0/3457039463 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f019819ca00 con 0x7f019810fe20 2024-04-28T12:50:39.164 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.209+0000 7f8aa4a00700 1 -- 172.21.15.106:0/2379279100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8aa01094a0 msgr2=0x7f8aa0109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.209+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/2379279100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8aa01094a0 0x7f8aa0109880 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009990 tx=0x7f8a90009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.209+0000 7f8aa4a00700 1 -- 172.21.15.106:0/2379279100 shutdown_connections 2024-04-28T12:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.209+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/2379279100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8aa006bad0 0x7f8aa01102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.289 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.209+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/2379279100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8aa01094a0 0x7f8aa0109880 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.209+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/2379279100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8aa006b130 0x7f8aa006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.209+0000 7f8aa4a00700 1 -- 172.21.15.106:0/2379279100 >> 172.21.15.106:0/2379279100 conn(0x7f8aa00792d0 msgr2=0x7f8aa00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.209+0000 7f8aa4a00700 1 -- 172.21.15.106:0/2379279100 shutdown_connections 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.209+0000 7f8aa4a00700 1 -- 172.21.15.106:0/2379279100 wait complete. 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.210+0000 7f8aa4a00700 1 Processor -- start 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.210+0000 7f8aa4a00700 1 -- start start 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8aa4a00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8aa006b130 0x7f8aa019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8aa4a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8aa006bad0 0x7f8aa019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8aa4a00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8aa01094a0 0x7f8aa0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8aa4a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0113650 con 0x7f8aa006bad0 2024-04-28T12:50:39.290 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8aa4a00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa01134d0 con 0x7f8aa01094a0 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8aa4a00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa01137d0 con 0x7f8aa006b130 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9d400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8aa006bad0 0x7f8aa019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8aa006b130 0x7f8aa019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9e800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8aa01094a0 0x7f8aa0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9de00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8aa006b130 0x7f8aa019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:43778/0 (socket says 172.21.15.106:43778) 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9de00700 1 -- 172.21.15.106:0/1317113682 learned_addr learned my addr 172.21.15.106:0/1317113682 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9e800700 1 -- 172.21.15.106:0/1317113682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8aa006b130 msgr2=0x7f8aa019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9e800700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8aa006b130 0x7f8aa019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9e800700 1 -- 172.21.15.106:0/1317113682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8aa006bad0 msgr2=0x7f8aa019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9e800700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8aa006bad0 0x7f8aa019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.211+0000 7f8a9e800700 1 -- 172.21.15.106:0/1317113682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa019a0f0 con 0x7f8aa01094a0 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.212+0000 7f8a9e800700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8aa01094a0 0x7f8aa0199990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800b330 tx=0x7f8a9800b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.212+0000 7f8a96a00700 1 -- 172.21.15.106:0/1317113682 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800bdd0 con 0x7f8aa01094a0 2024-04-28T12:50:39.291 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.212+0000 7f8a96a00700 1 -- 172.21.15.106:0/1317113682 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a98004bb0 con 0x7f8aa01094a0 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.212+0000 7f8a96a00700 1 -- 172.21.15.106:0/1317113682 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98007dc0 con 0x7f8aa01094a0 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.212+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa019a380 con 0x7f8aa01094a0 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.212+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa0102320 con 0x7f8aa01094a0 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.214+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa0104b50 con 0x7f8aa01094a0 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.215+0000 7f8a96a00700 1 -- 172.21.15.106:0/1317113682 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8a98020020 con 0x7f8aa01094a0 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.217+0000 7f8a96a00700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8a840a3d80 0x7f8a840a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.217+0000 7f8a96a00700 1 -- 172.21.15.106:0/1317113682 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f8a980d3e50 con 0x7f8aa01094a0 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.217+0000 7f8a9de00700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8a840a3d80 0x7f8a840a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.218+0000 7f8a9de00700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8a840a3d80 0x7f8a840a6240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a90006010 tx=0x7f8a9000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:39.292 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.221+0000 7f8a96a00700 1 -- 172.21.15.106:0/1317113682 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8a980a0390 con 0x7f8aa01094a0 2024-04-28T12:50:39.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.357+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f019819cfe0 con 0x7f019810fe20 2024-04-28T12:50:39.354 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:39 smithi106 ceph-mon[28081]: pgmap v372: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:50:39.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.358+0000 7f0196000700 1 -- 172.21.15.106:0/3457039463 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0190022020 con 0x7f019810fe20 2024-04-28T12:50:39.355 INFO:teuthology.orchestra.run.smithi106.stdout:107374182480 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f01800a3cd0 msgr2=0x7f01800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f01800a3cd0 0x7f01800a6190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f018c006010 tx=0x7f018c00b280 comp rx=0 tx=0).stop 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f019810fe20 msgr2=0x7f019819bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f019810fe20 0x7f019819bb20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f019000ef90 tx=0x7f019000c3a0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 shutdown_connections 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f019810fe20 0x7f019819bb20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0198075650 0x7f01981a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f01800a3cd0 0x7f01800a6190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 --2- 172.21.15.106:0/3457039463 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0198075f70 0x7f01981a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 >> 172.21.15.106:0/3457039463 conn(0x7f019807a580 msgr2=0x7f0198100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:39.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 shutdown_connections 2024-04-28T12:50:39.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.361+0000 7f01a0200700 1 -- 172.21.15.106:0/3457039463 wait complete. 2024-04-28T12:50:39.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.456+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8aa002e490 con 0x7f8aa01094a0 2024-04-28T12:50:39.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.457+0000 7f8a96a00700 1 -- 172.21.15.106:0/1317113682 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a9809fc90 con 0x7f8aa01094a0 2024-04-28T12:50:39.454 INFO:teuthology.orchestra.run.smithi106.stdout:55834574938 2024-04-28T12:50:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.460+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8a840a3d80 msgr2=0x7f8a840a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.460+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8a840a3d80 0x7f8a840a6240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8a90006010 tx=0x7f8a9000b280 comp rx=0 tx=0).stop 2024-04-28T12:50:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.460+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8aa01094a0 msgr2=0x7f8aa0199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.460+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8aa01094a0 0x7f8aa0199990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800b330 tx=0x7f8a9800b640 comp rx=0 tx=0).stop 2024-04-28T12:50:39.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.461+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 shutdown_connections 2024-04-28T12:50:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.461+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8aa01094a0 0x7f8aa0199990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.461+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8a840a3d80 0x7f8a840a6240 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.461+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8aa006b130 0x7f8aa019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.461+0000 7f8aa4a00700 1 --2- 172.21.15.106:0/1317113682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8aa006bad0 0x7f8aa019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.461+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 >> 172.21.15.106:0/1317113682 conn(0x7f8aa00792d0 msgr2=0x7f8aa0100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.461+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 shutdown_connections 2024-04-28T12:50:39.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:39.461+0000 7f8aa4a00700 1 -- 172.21.15.106:0/1317113682 wait complete. 2024-04-28T12:50:39.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:39 smithi171 ceph-mon[63127]: pgmap v372: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:50:39.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:39 smithi129 ceph-mon[62418]: pgmap v372: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:50:40.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:40 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3457039463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-28T12:50:40.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:40 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1317113682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-28T12:50:40.513 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:40 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3457039463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-28T12:50:40.513 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:40 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1317113682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-28T12:50:40.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:40 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3457039463' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-28T12:50:40.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:40 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1317113682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.858+0000 7f7a24e00700 1 -- 172.21.15.106:0/419870527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 msgr2=0x7f7a20112530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.858+0000 7f7a24e00700 1 --2- 172.21.15.106:0/419870527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 0x7f7a20112530 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800b0a0 tx=0x7f7a1800b3b0 comp rx=0 tx=0).stop 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.858+0000 7f7a24e00700 1 -- 172.21.15.106:0/419870527 shutdown_connections 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.858+0000 7f7a24e00700 1 --2- 172.21.15.106:0/419870527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 0x7f7a20112530 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.858+0000 7f7a24e00700 1 --2- 172.21.15.106:0/419870527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a2010b660 0x7f7a2010ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.858+0000 7f7a24e00700 1 --2- 172.21.15.106:0/419870527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a20105660 0x7f7a20105ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.858+0000 7f7a24e00700 1 -- 172.21.15.106:0/419870527 >> 172.21.15.106:0/419870527 conn(0x7f7a20101150 msgr2=0x7f7a20103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.858+0000 7f7a24e00700 1 -- 172.21.15.106:0/419870527 shutdown_connections 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.858+0000 7f7a24e00700 1 -- 172.21.15.106:0/419870527 wait complete. 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.859+0000 7f7a24e00700 1 Processor -- start 2024-04-28T12:50:40.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.859+0000 7f7a24e00700 1 -- start start 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a24e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a20105660 0x7f7a201a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a24e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 0x7f7a201a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a24e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a2010b660 0x7f7a2019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a24e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201158f0 con 0x7f7a20106020 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a24e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20115770 con 0x7f7a20105660 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a24e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20115a70 con 0x7f7a2010b660 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a1d400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 0x7f7a201a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a1d400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 0x7f7a201a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35302/0 (socket says 172.21.15.106:35302) 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a1d400700 1 -- 172.21.15.106:0/645107238 learned_addr learned my addr 172.21.15.106:0/645107238 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a1de00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a20105660 0x7f7a201a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.860+0000 7f7a1e800700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a2010b660 0x7f7a2019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a1d400700 1 -- 172.21.15.106:0/645107238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a2010b660 msgr2=0x7f7a2019bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:40.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a1d400700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a2010b660 0x7f7a2019bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a1d400700 1 -- 172.21.15.106:0/645107238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a20105660 msgr2=0x7f7a201a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a1d400700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a20105660 0x7f7a201a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a1d400700 1 -- 172.21.15.106:0/645107238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2019c410 con 0x7f7a20106020 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a1de00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a20105660 0x7f7a201a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a1d400700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 0x7f7a201a1fa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00e9c0 tx=0x7f7a0c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a16a00700 1 -- 172.21.15.106:0/645107238 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00cbd0 con 0x7f7a20106020 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a16a00700 1 -- 172.21.15.106:0/645107238 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a0c00ee60 con 0x7f7a20106020 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2019c700 con 0x7f7a20106020 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a16a00700 1 -- 172.21.15.106:0/645107238 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0055c0 con 0x7f7a20106020 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.861+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a201a89e0 con 0x7f7a20106020 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.863+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a20106da0 con 0x7f7a20106020 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.865+0000 7f7a16a00700 1 -- 172.21.15.106:0/645107238 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7a0c004750 con 0x7f7a20106020 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.867+0000 7f7a16a00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7a040a3d50 0x7f7a040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:40.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.867+0000 7f7a16a00700 1 -- 172.21.15.106:0/645107238 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f7a0c0d0b10 con 0x7f7a20106020 2024-04-28T12:50:40.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.867+0000 7f7a1de00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7a040a3d50 0x7f7a040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:40.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.868+0000 7f7a1de00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7a040a3d50 0x7f7a040a6210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009fd0 tx=0x7f7a10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:40.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:40.871+0000 7f7a16a00700 1 -- 172.21.15.106:0/645107238 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7a0c09cff0 con 0x7f7a20106020 2024-04-28T12:50:41.102 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.106+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f7a2019cff0 con 0x7f7a20106020 2024-04-28T12:50:41.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.106+0000 7f7a16a00700 1 -- 172.21.15.106:0/645107238 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7a0c09c8f0 con 0x7f7a20106020 2024-04-28T12:50:41.103 INFO:teuthology.orchestra.run.smithi106.stdout:343597383715 2024-04-28T12:50:41.106 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7a040a3d50 msgr2=0x7f7a040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:41.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7a040a3d50 0x7f7a040a6210 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009fd0 tx=0x7f7a10009260 comp rx=0 tx=0).stop 2024-04-28T12:50:41.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 msgr2=0x7f7a201a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:41.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 0x7f7a201a1fa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00e9c0 tx=0x7f7a0c00ecd0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 shutdown_connections 2024-04-28T12:50:41.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7a040a3d50 0x7f7a040a6210 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a2010b660 0x7f7a2019bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.107 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a20105660 0x7f7a201a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 --2- 172.21.15.106:0/645107238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a20106020 0x7f7a201a1fa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.110+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 >> 172.21.15.106:0/645107238 conn(0x7f7a20101150 msgr2=0x7f7a20101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:41.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.111+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 shutdown_connections 2024-04-28T12:50:41.108 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.111+0000 7f7a24e00700 1 -- 172.21.15.106:0/645107238 wait complete. 2024-04-28T12:50:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.224+0000 7f5dba000700 1 -- 172.21.15.106:0/1228692768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db4103470 msgr2=0x7f5db41038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.224+0000 7f5dba000700 1 --2- 172.21.15.106:0/1228692768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db4103470 0x7f5db41038f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009990 tx=0x7f5d9c009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.225+0000 7f5dba000700 1 -- 172.21.15.106:0/1228692768 shutdown_connections 2024-04-28T12:50:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.225+0000 7f5dba000700 1 --2- 172.21.15.106:0/1228692768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db4103e30 0x7f5db4110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.225+0000 7f5dba000700 1 --2- 172.21.15.106:0/1228692768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db41094c0 0x7f5db41098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.225+0000 7f5dba000700 1 --2- 172.21.15.106:0/1228692768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db4103470 0x7f5db41038f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.225+0000 7f5dba000700 1 -- 172.21.15.106:0/1228692768 >> 172.21.15.106:0/1228692768 conn(0x7f5db407a580 msgr2=0x7f5db407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.225+0000 7f5dba000700 1 -- 172.21.15.106:0/1228692768 shutdown_connections 2024-04-28T12:50:41.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.225+0000 7f5dba000700 1 -- 172.21.15.106:0/1228692768 wait complete. 2024-04-28T12:50:41.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.226+0000 7f5dba000700 1 Processor -- start 2024-04-28T12:50:41.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.227+0000 7f5dba000700 1 -- start start 2024-04-28T12:50:41.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.227+0000 7f5dba000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4103470 0x7f5db419bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:41.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.227+0000 7f5dba000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db4103e30 0x7f5db419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:41.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.227+0000 7f5dba000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db41094c0 0x7f5db41a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:41.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.227+0000 7f5dba000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4113610 con 0x7f5db4103e30 2024-04-28T12:50:41.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.227+0000 7f5dba000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4113490 con 0x7f5db41094c0 2024-04-28T12:50:41.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.227+0000 7f5dba000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4113790 con 0x7f5db4103470 2024-04-28T12:50:41.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.227+0000 7f5db3400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4103470 0x7f5db419bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:41.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5dabe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db4103e30 0x7f5db419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5db3400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4103470 0x7f5db419bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:43814/0 (socket says 172.21.15.106:43814) 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5db3400700 1 -- 172.21.15.106:0/1524063291 learned_addr learned my addr 172.21.15.106:0/1524063291 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5db3e00700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db41094c0 0x7f5db41a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5db3400700 1 -- 172.21.15.106:0/1524063291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db41094c0 msgr2=0x7f5db41a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5db3400700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db41094c0 0x7f5db41a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5db3400700 1 -- 172.21.15.106:0/1524063291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db4103e30 msgr2=0x7f5db419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5db3400700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db4103e30 0x7f5db419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5db3400700 1 -- 172.21.15.106:0/1524063291 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db41a0d90 con 0x7f5db4103470 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.228+0000 7f5db3400700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4103470 0x7f5db419bd80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5da400d980 tx=0x7f5da400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.229+0000 7f5db0c00700 1 -- 172.21.15.106:0/1524063291 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4016070 con 0x7f5db4103470 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.229+0000 7f5db0c00700 1 -- 172.21.15.106:0/1524063291 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da4004830 con 0x7f5db4103470 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.229+0000 7f5db0c00700 1 -- 172.21.15.106:0/1524063291 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4005540 con 0x7f5db4103470 2024-04-28T12:50:41.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.229+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db41a1080 con 0x7f5db4103470 2024-04-28T12:50:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.229+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db41a8900 con 0x7f5db4103470 2024-04-28T12:50:41.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.230+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db406a110 con 0x7f5db4103470 2024-04-28T12:50:41.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.233+0000 7f5db0c00700 1 -- 172.21.15.106:0/1524063291 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5da4004220 con 0x7f5db4103470 2024-04-28T12:50:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.235+0000 7f5db0c00700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5d980a3d50 0x7f5d980a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:41.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.235+0000 7f5db0c00700 1 -- 172.21.15.106:0/1524063291 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f5da40d05a0 con 0x7f5db4103470 2024-04-28T12:50:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.235+0000 7f5dabe00700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5d980a3d50 0x7f5d980a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:41.232 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.236+0000 7f5dabe00700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5d980a3d50 0x7f5d980a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5db407ae20 tx=0x7f5d9c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:41.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.239+0000 7f5db0c00700 1 -- 172.21.15.106:0/1524063291 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5da401f020 con 0x7f5db4103470 2024-04-28T12:50:41.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.478+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5db402e650 con 0x7f5db4103470 2024-04-28T12:50:41.475 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:41 smithi106 ceph-mon[28081]: pgmap v373: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:50:41.475 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:41 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/645107238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-28T12:50:41.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.479+0000 7f5db0c00700 1 -- 172.21.15.106:0/1524063291 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5da4022350 con 0x7f5db4103470 2024-04-28T12:50:41.476 INFO:teuthology.orchestra.run.smithi106.stdout:163208757317 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.481+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5d980a3d50 msgr2=0x7f5d980a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5d980a3d50 0x7f5d980a6210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5db407ae20 tx=0x7f5d9c00b540 comp rx=0 tx=0).stop 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4103470 msgr2=0x7f5db419bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4103470 0x7f5db419bd80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5da400d980 tx=0x7f5da400dc90 comp rx=0 tx=0).stop 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 shutdown_connections 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5db41094c0 0x7f5db41a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5d980a3d50 0x7f5d980a6210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5db4103470 0x7f5db419bd80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 --2- 172.21.15.106:0/1524063291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5db4103e30 0x7f5db419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 >> 172.21.15.106:0/1524063291 conn(0x7f5db407a580 msgr2=0x7f5db410e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 shutdown_connections 2024-04-28T12:50:41.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:41.482+0000 7f5dba000700 1 -- 172.21.15.106:0/1524063291 wait complete. 2024-04-28T12:50:41.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:41 smithi171 ceph-mon[63127]: pgmap v373: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:50:41.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:41 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/645107238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-28T12:50:41.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:41 smithi129 ceph-mon[62418]: pgmap v373: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:50:41.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:41 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/645107238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-28T12:50:42.084 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411409 got 77309411414 for osd.2 2024-04-28T12:50:42.085 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:42.200 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332148 got 219043332153 for osd.7 2024-04-28T12:50:42.200 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:42.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:42 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1524063291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-28T12:50:42.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:42 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1524063291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-28T12:50:42.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:42 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1524063291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-28T12:50:43.125 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383706 got 343597383715 for osd.11 2024-04-28T12:50:43.125 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:43.314 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:43 smithi106 ceph-mon[28081]: pgmap v374: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:50:43.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:43 smithi171 ceph-mon[63127]: pgmap v374: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:50:43.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:43 smithi129 ceph-mon[62418]: pgmap v374: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:50:44.262 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574930 got 55834574938 for osd.1 2024-04-28T12:50:44.262 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:44.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:44 smithi106 ceph-mon[28081]: pgmap v375: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:50:44.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:44 smithi171 ceph-mon[63127]: pgmap v375: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:50:44.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:44 smithi129 ceph-mon[62418]: pgmap v375: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:50:44.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757310 got 163208757317 for osd.5 2024-04-28T12:50:44.687 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:45.272 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182473 got 107374182480 for osd.3 2024-04-28T12:50:45.272 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:46.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.342+0000 7ffa52400700 1 -- 172.21.15.106:0/269920592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa4c06b670 msgr2=0x7ffa4c06bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:46.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.342+0000 7ffa52400700 1 --2- 172.21.15.106:0/269920592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa4c06b670 0x7ffa4c06bad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009990 tx=0x7ffa38009ca0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.342+0000 7ffa52400700 1 -- 172.21.15.106:0/269920592 shutdown_connections 2024-04-28T12:50:46.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.342+0000 7ffa52400700 1 --2- 172.21.15.106:0/269920592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa4c105e60 0x7ffa4c1062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.339 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.342+0000 7ffa52400700 1 --2- 172.21.15.106:0/269920592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa4c108350 0x7ffa4c06b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.342+0000 7ffa52400700 1 --2- 172.21.15.106:0/269920592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa4c06b670 0x7ffa4c06bad0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.342+0000 7ffa52400700 1 -- 172.21.15.106:0/269920592 >> 172.21.15.106:0/269920592 conn(0x7ffa4c0792d0 msgr2=0x7ffa4c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:46.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.342+0000 7ffa52400700 1 -- 172.21.15.106:0/269920592 shutdown_connections 2024-04-28T12:50:46.340 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.342+0000 7ffa52400700 1 -- 172.21.15.106:0/269920592 wait complete. 2024-04-28T12:50:46.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.344+0000 7ffa52400700 1 Processor -- start 2024-04-28T12:50:46.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.344+0000 7ffa52400700 1 -- start start 2024-04-28T12:50:46.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.344+0000 7ffa52400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa4c06b670 0x7ffa4c1a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:46.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.344+0000 7ffa52400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa4c105e60 0x7ffa4c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:46.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.344+0000 7ffa52400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa4c108350 0x7ffa4c1a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:46.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.344+0000 7ffa52400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c119c00 con 0x7ffa4c108350 2024-04-28T12:50:46.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.344+0000 7ffa52400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c119a80 con 0x7ffa4c06b670 2024-04-28T12:50:46.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.344+0000 7ffa52400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c119d80 con 0x7ffa4c105e60 2024-04-28T12:50:46.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4aa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa4c105e60 0x7ffa4c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:46.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4aa00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa4c105e60 0x7ffa4c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:43838/0 (socket says 172.21.15.106:43838) 2024-04-28T12:50:46.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4aa00700 1 -- 172.21.15.106:0/3568593721 learned_addr learned my addr 172.21.15.106:0/3568593721 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:46.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4b400700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa4c06b670 0x7ffa4c1a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:46.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4be00700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa4c108350 0x7ffa4c1a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:46.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4aa00700 1 -- 172.21.15.106:0/3568593721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa4c06b670 msgr2=0x7ffa4c1a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:46.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4aa00700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa4c06b670 0x7ffa4c1a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4aa00700 1 -- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa4c108350 msgr2=0x7ffa4c1a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:46.343 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4aa00700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa4c108350 0x7ffa4c1a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4aa00700 1 -- 172.21.15.106:0/3568593721 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c1a5140 con 0x7ffa4c105e60 2024-04-28T12:50:46.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.345+0000 7ffa4be00700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa4c108350 0x7ffa4c1a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:50:46.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.346+0000 7ffa4aa00700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa4c105e60 0x7ffa4c1a0670 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000c00 tx=0x7ffa380057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:46.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.346+0000 7ffa43e00700 1 -- 172.21.15.106:0/3568593721 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38027070 con 0x7ffa4c105e60 2024-04-28T12:50:46.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.346+0000 7ffa43e00700 1 -- 172.21.15.106:0/3568593721 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa38022470 con 0x7ffa4c105e60 2024-04-28T12:50:46.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.346+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c1a53d0 con 0x7ffa4c105e60 2024-04-28T12:50:46.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.346+0000 7ffa43e00700 1 -- 172.21.15.106:0/3568593721 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38017650 con 0x7ffa4c105e60 2024-04-28T12:50:46.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.346+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c1accb0 con 0x7ffa4c105e60 2024-04-28T12:50:46.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.347+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4c06a110 con 0x7ffa4c105e60 2024-04-28T12:50:46.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.349+0000 7ffa43e00700 1 -- 172.21.15.106:0/3568593721 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ffa3802a070 con 0x7ffa4c105e60 2024-04-28T12:50:46.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.351+0000 7ffa43e00700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffa300a3cd0 0x7ffa300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:46.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.351+0000 7ffa43e00700 1 -- 172.21.15.106:0/3568593721 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7ffa380d1620 con 0x7ffa4c105e60 2024-04-28T12:50:46.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.351+0000 7ffa4b400700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffa300a3cd0 0x7ffa300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:46.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.352+0000 7ffa4b400700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffa300a3cd0 0x7ffa300a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009be0 tx=0x7ffa3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:46.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.355+0000 7ffa43e00700 1 -- 172.21.15.106:0/3568593721 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ffa38013070 con 0x7ffa4c105e60 2024-04-28T12:50:46.581 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.584+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ffa4c1ace80 con 0x7ffa4c105e60 2024-04-28T12:50:46.582 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.585+0000 7ffa43e00700 1 -- 172.21.15.106:0/3568593721 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa3809db90 con 0x7ffa4c105e60 2024-04-28T12:50:46.582 INFO:teuthology.orchestra.run.smithi106.stdout:34359738466 2024-04-28T12:50:46.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffa300a3cd0 msgr2=0x7ffa300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:46.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffa300a3cd0 0x7ffa300a6190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009be0 tx=0x7ffa3c009260 comp rx=0 tx=0).stop 2024-04-28T12:50:46.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa4c105e60 msgr2=0x7ffa4c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:46.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa4c105e60 0x7ffa4c1a0670 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000c00 tx=0x7ffa380057c0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 shutdown_connections 2024-04-28T12:50:46.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa4c108350 0x7ffa4c1a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa4c06b670 0x7ffa4c1a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ffa300a3cd0 0x7ffa300a6190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 --2- 172.21.15.106:0/3568593721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa4c105e60 0x7ffa4c1a0670 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:46.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 >> 172.21.15.106:0/3568593721 conn(0x7ffa4c0792d0 msgr2=0x7ffa4c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:46.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 shutdown_connections 2024-04-28T12:50:46.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:46.588+0000 7ffa52400700 1 -- 172.21.15.106:0/3568593721 wait complete. 2024-04-28T12:50:47.308 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:47 smithi106 ceph-mon[28081]: pgmap v376: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T12:50:47.308 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:50:47.308 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:47 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:50:47.308 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:47 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3568593721' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-28T12:50:47.364 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738460 got 34359738466 for osd.0 2024-04-28T12:50:47.364 DEBUG:teuthology.parallel:result is None 2024-04-28T12:50:47.364 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-28T12:50:47.365 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-28T12:50:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:47 smithi171 ceph-mon[63127]: pgmap v376: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T12:50:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:50:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:47 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:50:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:47 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3568593721' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-28T12:50:47.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:47 smithi129 ceph-mon[62418]: pgmap v376: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T12:50:47.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:50:47.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:47 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:50:47.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:47 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3568593721' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-28T12:50:47.787 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:48.288 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:50:48.288 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:50:48.288 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:50:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:50:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:50:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:50:48.513 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:50:48.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:50:48.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:50:49.415 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:49 smithi129 ceph-mon[62418]: pgmap v377: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:50:49.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:49 smithi171 ceph-mon[63127]: pgmap v377: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:50:49.504 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:49 smithi106 ceph-mon[28081]: pgmap v377: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:50:49.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.990+0000 7f703be00700 1 -- 172.21.15.106:0/801766782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410a410 msgr2=0x7f7034116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:49.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.990+0000 7f703be00700 1 --2- 172.21.15.106:0/801766782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410a410 0x7f7034116920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f702c00b0a0 tx=0x7f702c00b3b0 comp rx=0 tx=0).stop 2024-04-28T12:50:49.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.990+0000 7f703be00700 1 -- 172.21.15.106:0/801766782 shutdown_connections 2024-04-28T12:50:49.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.990+0000 7f703be00700 1 --2- 172.21.15.106:0/801766782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410a410 0x7f7034116920 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:49.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.990+0000 7f703be00700 1 --2- 172.21.15.106:0/801766782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f703410fa50 0x7f703410fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:49.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.990+0000 7f703be00700 1 --2- 172.21.15.106:0/801766782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7034109a50 0x7f7034109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:49.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.990+0000 7f703be00700 1 -- 172.21.15.106:0/801766782 >> 172.21.15.106:0/801766782 conn(0x7f7034076bf0 msgr2=0x7f7034108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:49.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.991+0000 7f703be00700 1 -- 172.21.15.106:0/801766782 shutdown_connections 2024-04-28T12:50:49.988 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.991+0000 7f703be00700 1 -- 172.21.15.106:0/801766782 wait complete. 2024-04-28T12:50:49.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.992+0000 7f703be00700 1 Processor -- start 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.992+0000 7f703be00700 1 -- start start 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f703be00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7034109a50 0x7f70341a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f703be00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f703410a410 0x7f70341a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f703be00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410fa50 0x7f70341a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f703be00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7034119ca0 con 0x7f703410fa50 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f703be00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7034119b20 con 0x7f703410a410 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f703be00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7034119e20 con 0x7f7034109a50 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7039e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410fa50 0x7f70341a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7039e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410fa50 0x7f70341a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52946/0 (socket says 172.21.15.106:52946) 2024-04-28T12:50:49.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7039e00700 1 -- 172.21.15.106:0/176667904 learned_addr learned my addr 172.21.15.106:0/176667904 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:49.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7039400700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7034109a50 0x7f70341a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:49.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7038a00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f703410a410 0x7f70341a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7039e00700 1 -- 172.21.15.106:0/176667904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7034109a50 msgr2=0x7f70341a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7039e00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7034109a50 0x7f70341a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7039e00700 1 -- 172.21.15.106:0/176667904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f703410a410 msgr2=0x7f70341a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7039e00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f703410a410 0x7f70341a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.993+0000 7f7039e00700 1 -- 172.21.15.106:0/176667904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70341a5180 con 0x7f703410fa50 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.994+0000 7f7039e00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410fa50 0x7f70341a4a40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f702c00b070 tx=0x7f702c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.994+0000 7f7032000700 1 -- 172.21.15.106:0/176667904 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c021070 con 0x7f703410fa50 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.994+0000 7f7032000700 1 -- 172.21.15.106:0/176667904 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f702c004e40 con 0x7f703410fa50 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.994+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70341a5410 con 0x7f703410fa50 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.994+0000 7f7032000700 1 -- 172.21.15.106:0/176667904 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c004630 con 0x7f703410fa50 2024-04-28T12:50:49.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.994+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70341accf0 con 0x7f703410fa50 2024-04-28T12:50:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.996+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703406a110 con 0x7f703410fa50 2024-04-28T12:50:49.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:49.999+0000 7f7032000700 1 -- 172.21.15.106:0/176667904 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f702c007980 con 0x7f703410fa50 2024-04-28T12:50:49.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.000+0000 7f7032000700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f701c0a3cd0 0x7f701c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:49.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.001+0000 7f7032000700 1 -- 172.21.15.106:0/176667904 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7f702c00f020 con 0x7f703410fa50 2024-04-28T12:50:49.997 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.001+0000 7f7039400700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f701c0a3cd0 0x7f701c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:49.998 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.002+0000 7f7039400700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f701c0a3cd0 0x7f701c0a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7028009be0 tx=0x7f7028009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:50.000 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.003+0000 7f7032000700 1 -- 172.21.15.106:0/176667904 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f702c02a020 con 0x7f703410fa50 2024-04-28T12:50:50.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.225+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70341a14d0 con 0x7f701c0a3cd0 2024-04-28T12:50:50.253 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.256+0000 7f7032000700 1 -- 172.21.15.106:0/176667904 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340414 (secure 0 0 0) 0x7f70341a14d0 con 0x7f701c0a3cd0 2024-04-28T12:50:50.262 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.265+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f701c0a3cd0 msgr2=0x7f701c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.265+0000 7f703be00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f701c0a3cd0 0x7f701c0a6190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7028009be0 tx=0x7f7028009260 comp rx=0 tx=0).stop 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.265+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410fa50 msgr2=0x7f70341a4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.265+0000 7f703be00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410fa50 0x7f70341a4a40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f702c00b070 tx=0x7f702c009c90 comp rx=0 tx=0).stop 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.266+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 shutdown_connections 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.266+0000 7f703be00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703410fa50 0x7f70341a4a40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.266+0000 7f703be00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f701c0a3cd0 0x7f701c0a6190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.266+0000 7f703be00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7034109a50 0x7f70341a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.266+0000 7f703be00700 1 --2- 172.21.15.106:0/176667904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f703410a410 0x7f70341a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.266+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 >> 172.21.15.106:0/176667904 conn(0x7f7034076bf0 msgr2=0x7f7034077210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.266+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 shutdown_connections 2024-04-28T12:50:50.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:50.266+0000 7f703be00700 1 -- 172.21.15.106:0/176667904 wait complete. 2024-04-28T12:50:50.266 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-28T12:50:51.095 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-04-28T12:50:50.103626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649968,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":933,"num_read_kb":577,"num_write":425,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367600,"kb_used_data":14204,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574224,"statfs":{"total":1151940427776,"available":1151564005376,"internally_reserved":0,"allocated":14544896,"data_stored":7934366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006566"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195899+0000","last_change":"2024-04-28T12:48:54.979081+0000","last_active":"2024-04-28T12:49:57.195899+0000","last_peered":"2024-04-28T12:49:57.195899+0000","last_clean":"2024-04-28T12:49:57.195899+0000","last_became_active":"2024-04-28T12:48:54.978668+0000","last_became_peered":"2024-04-28T12:48:54.978668+0000","last_unstale":"2024-04-28T12:49:57.195899+0000","last_undegraded":"2024-04-28T12:49:57.195899+0000","last_fullsized":"2024-04-28T12:49:57.195899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:43.345706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.201410+0000","last_change":"2024-04-28T12:48:54.966277+0000","last_active":"2024-04-28T12:49:57.201410+0000","last_peered":"2024-04-28T12:49:57.201410+0000","last_clean":"2024-04-28T12:49:57.201410+0000","last_became_active":"2024-04-28T12:48:54.965880+0000","last_became_peered":"2024-04-28T12:48:54.965880+0000","last_unstale":"2024-04-28T12:49:57.201410+0000","last_undegraded":"2024-04-28T12:49:57.201410+0000","last_fullsized":"2024-04-28T12:49:57.201410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:42:13.166459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.201177+0000","last_change":"2024-04-28T12:48:54.966516+0000","last_active":"2024-04-28T12:49:57.201177+0000","last_peered":"2024-04-28T12:49:57.201177+0000","last_clean":"2024-04-28T12:49:57.201177+0000","last_became_active":"2024-04-28T12:48:54.966112+0000","last_became_peered":"2024-04-28T12:48:54.966112+0000","last_unstale":"2024-04-28T12:49:57.201177+0000","last_undegraded":"2024-04-28T12:49:57.201177+0000","last_fullsized":"2024-04-28T12:49:57.201177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:15:05.538389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193943+0000","last_change":"2024-04-28T12:48:54.962718+0000","last_active":"2024-04-28T12:49:57.193943+0000","last_peered":"2024-04-28T12:49:57.193943+0000","last_clean":"2024-04-28T12:49:57.193943+0000","last_became_active":"2024-04-28T12:48:54.962371+0000","last_became_peered":"2024-04-28T12:48:54.962371+0000","last_unstale":"2024-04-28T12:49:57.193943+0000","last_undegraded":"2024-04-28T12:49:57.193943+0000","last_fullsized":"2024-04-28T12:49:57.193943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:55:47.714734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197823+0000","last_change":"2024-04-28T12:48:54.986898+0000","last_active":"2024-04-28T12:49:57.197823+0000","last_peered":"2024-04-28T12:49:57.197823+0000","last_clean":"2024-04-28T12:49:57.197823+0000","last_became_active":"2024-04-28T12:48:54.986550+0000","last_became_peered":"2024-04-28T12:48:54.986550+0000","last_unstale":"2024-04-28T12:49:57.197823+0000","last_undegraded":"2024-04-28T12:49:57.197823+0000","last_fullsized":"2024-04-28T12:49:57.197823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:54:54.942449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436383+0000","last_change":"2024-04-28T12:48:54.981357+0000","last_active":"2024-04-28T12:49:57.436383+0000","last_peered":"2024-04-28T12:49:57.436383+0000","last_clean":"2024-04-28T12:49:57.436383+0000","last_became_active":"2024-04-28T12:48:54.980945+0000","last_became_peered":"2024-04-28T12:48:54.980945+0000","last_unstale":"2024-04-28T12:49:57.436383+0000","last_undegraded":"2024-04-28T12:49:57.436383+0000","last_fullsized":"2024-04-28T12:49:57.436383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:21:51.476872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.439357+0000","last_change":"2024-04-28T12:49:57.439357+0000","last_active":"2024-04-28T12:49:57.439357+0000","last_peered":"2024-04-28T12:49:57.439357+0000","last_clean":"2024-04-28T12:49:57.439357+0000","last_became_active":"2024-04-28T12:49:57.438909+0000","last_became_peered":"2024-04-28T12:49:57.438909+0000","last_unstale":"2024-04-28T12:49:57.439357+0000","last_undegraded":"2024-04-28T12:49:57.439357+0000","last_fullsized":"2024-04-28T12:49:57.439357+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:34:45.679426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.211795+0000","last_change":"2024-04-28T12:48:54.979500+0000","last_active":"2024-04-28T12:49:57.211795+0000","last_peered":"2024-04-28T12:49:57.211795+0000","last_clean":"2024-04-28T12:49:57.211795+0000","last_became_active":"2024-04-28T12:48:54.978541+0000","last_became_peered":"2024-04-28T12:48:54.978541+0000","last_unstale":"2024-04-28T12:49:57.211795+0000","last_undegraded":"2024-04-28T12:49:57.211795+0000","last_fullsized":"2024-04-28T12:49:57.211795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:06:47.550461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.210318+0000","last_change":"2024-04-28T12:49:57.210318+0000","last_active":"2024-04-28T12:49:57.210318+0000","last_peered":"2024-04-28T12:49:57.210318+0000","last_clean":"2024-04-28T12:49:57.210318+0000","last_became_active":"2024-04-28T12:49:57.209987+0000","last_became_peered":"2024-04-28T12:49:57.209987+0000","last_unstale":"2024-04-28T12:49:57.210318+0000","last_undegraded":"2024-04-28T12:49:57.210318+0000","last_fullsized":"2024-04-28T12:49:57.210318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:08:27.279256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436725+0000","last_change":"2024-04-28T12:48:54.961425+0000","last_active":"2024-04-28T12:49:57.436725+0000","last_peered":"2024-04-28T12:49:57.436725+0000","last_clean":"2024-04-28T12:49:57.436725+0000","last_became_active":"2024-04-28T12:48:54.961060+0000","last_became_peered":"2024-04-28T12:48:54.961060+0000","last_unstale":"2024-04-28T12:49:57.436725+0000","last_undegraded":"2024-04-28T12:49:57.436725+0000","last_fullsized":"2024-04-28T12:49:57.436725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:49:14.184108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198106+0000","last_change":"2024-04-28T12:48:54.963483+0000","last_active":"2024-04-28T12:49:57.198106+0000","last_peered":"2024-04-28T12:49:57.198106+0000","last_clean":"2024-04-28T12:49:57.198106+0000","last_became_active":"2024-04-28T12:48:54.963146+0000","last_became_peered":"2024-04-28T12:48:54.963146+0000","last_unstale":"2024-04-28T12:49:57.198106+0000","last_undegraded":"2024-04-28T12:49:57.198106+0000","last_fullsized":"2024-04-28T12:49:57.198106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:05:30.313659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193857+0000","last_change":"2024-04-28T12:48:54.961834+0000","last_active":"2024-04-28T12:49:57.193857+0000","last_peered":"2024-04-28T12:49:57.193857+0000","last_clean":"2024-04-28T12:49:57.193857+0000","last_became_active":"2024-04-28T12:48:54.961521+0000","last_became_peered":"2024-04-28T12:48:54.961521+0000","last_unstale":"2024-04-28T12:49:57.193857+0000","last_undegraded":"2024-04-28T12:49:57.193857+0000","last_fullsized":"2024-04-28T12:49:57.193857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:34:57.168391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202673+0000","last_change":"2024-04-28T12:48:54.979673+0000","last_active":"2024-04-28T12:49:57.202673+0000","last_peered":"2024-04-28T12:49:57.202673+0000","last_clean":"2024-04-28T12:49:57.202673+0000","last_became_active":"2024-04-28T12:48:54.976440+0000","last_became_peered":"2024-04-28T12:48:54.976440+0000","last_unstale":"2024-04-28T12:49:57.202673+0000","last_undegraded":"2024-04-28T12:49:57.202673+0000","last_fullsized":"2024-04-28T12:49:57.202673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:46:30.391135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199191+0000","last_change":"2024-04-28T12:48:54.963108+0000","last_active":"2024-04-28T12:49:57.199191+0000","last_peered":"2024-04-28T12:49:57.199191+0000","last_clean":"2024-04-28T12:49:57.199191+0000","last_became_active":"2024-04-28T12:48:54.962294+0000","last_became_peered":"2024-04-28T12:48:54.962294+0000","last_unstale":"2024-04-28T12:49:57.199191+0000","last_undegraded":"2024-04-28T12:49:57.199191+0000","last_fullsized":"2024-04-28T12:49:57.199191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:10:03.438962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199269+0000","last_change":"2024-04-28T12:48:54.967990+0000","last_active":"2024-04-28T12:49:57.199269+0000","last_peered":"2024-04-28T12:49:57.199269+0000","last_clean":"2024-04-28T12:49:57.199269+0000","last_became_active":"2024-04-28T12:48:54.967608+0000","last_became_peered":"2024-04-28T12:48:54.967608+0000","last_unstale":"2024-04-28T12:49:57.199269+0000","last_undegraded":"2024-04-28T12:49:57.199269+0000","last_fullsized":"2024-04-28T12:49:57.199269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:24:17.209965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198739+0000","last_change":"2024-04-28T12:48:54.974436+0000","last_active":"2024-04-28T12:49:57.198739+0000","last_peered":"2024-04-28T12:49:57.198739+0000","last_clean":"2024-04-28T12:49:57.198739+0000","last_became_active":"2024-04-28T12:48:54.973950+0000","last_became_peered":"2024-04-28T12:48:54.973950+0000","last_unstale":"2024-04-28T12:49:57.198739+0000","last_undegraded":"2024-04-28T12:49:57.198739+0000","last_fullsized":"2024-04-28T12:49:57.198739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:51:08.957853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195685+0000","last_change":"2024-04-28T12:48:54.979452+0000","last_active":"2024-04-28T12:49:57.195685+0000","last_peered":"2024-04-28T12:49:57.195685+0000","last_clean":"2024-04-28T12:49:57.195685+0000","last_became_active":"2024-04-28T12:48:54.979033+0000","last_became_peered":"2024-04-28T12:48:54.979033+0000","last_unstale":"2024-04-28T12:49:57.195685+0000","last_undegraded":"2024-04-28T12:49:57.195685+0000","last_fullsized":"2024-04-28T12:49:57.195685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:26:46.644763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.438059+0000","last_change":"2024-04-28T12:48:54.970775+0000","last_active":"2024-04-28T12:49:57.438059+0000","last_peered":"2024-04-28T12:49:57.438059+0000","last_clean":"2024-04-28T12:49:57.438059+0000","last_became_active":"2024-04-28T12:48:54.970414+0000","last_became_peered":"2024-04-28T12:48:54.970414+0000","last_unstale":"2024-04-28T12:49:57.438059+0000","last_undegraded":"2024-04-28T12:49:57.438059+0000","last_fullsized":"2024-04-28T12:49:57.438059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:49:59.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193732+0000","last_change":"2024-04-28T12:48:54.983460+0000","last_active":"2024-04-28T12:49:57.193732+0000","last_peered":"2024-04-28T12:49:57.193732+0000","last_clean":"2024-04-28T12:49:57.193732+0000","last_became_active":"2024-04-28T12:48:54.983083+0000","last_became_peered":"2024-04-28T12:48:54.983083+0000","last_unstale":"2024-04-28T12:49:57.193732+0000","last_undegraded":"2024-04-28T12:49:57.193732+0000","last_fullsized":"2024-04-28T12:49:57.193732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:56:14.922227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193769+0000","last_change":"2024-04-28T12:48:59.198781+0000","last_active":"2024-04-28T12:49:57.193769+0000","last_peered":"2024-04-28T12:49:57.193769+0000","last_clean":"2024-04-28T12:49:57.193769+0000","last_became_active":"2024-04-28T12:48:59.198393+0000","last_became_peered":"2024-04-28T12:48:59.198393+0000","last_unstale":"2024-04-28T12:49:57.193769+0000","last_undegraded":"2024-04-28T12:49:57.193769+0000","last_fullsized":"2024-04-28T12:49:57.193769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:37:01.888857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:47.206103+0000","last_change":"2024-04-28T12:48:54.131557+0000","last_active":"2024-04-28T12:50:47.206103+0000","last_peered":"2024-04-28T12:50:47.206103+0000","last_clean":"2024-04-28T12:50:47.206103+0000","last_became_active":"2024-04-28T12:48:54.128561+0000","last_became_peered":"2024-04-28T12:48:54.128561+0000","last_unstale":"2024-04-28T12:50:47.206103+0000","last_undegraded":"2024-04-28T12:50:47.206103+0000","last_fullsized":"2024-04-28T12:50:47.206103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:19:41.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.212442+0000","last_change":"2024-04-28T12:48:54.978972+0000","last_active":"2024-04-28T12:49:57.212442+0000","last_peered":"2024-04-28T12:49:57.212442+0000","last_clean":"2024-04-28T12:49:57.212442+0000","last_became_active":"2024-04-28T12:48:54.977025+0000","last_became_peered":"2024-04-28T12:48:54.977025+0000","last_unstale":"2024-04-28T12:49:57.212442+0000","last_undegraded":"2024-04-28T12:49:57.212442+0000","last_fullsized":"2024-04-28T12:49:57.212442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:22:54.411569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193436+0000","last_change":"2024-04-28T12:48:59.193410+0000","last_active":"2024-04-28T12:49:57.193436+0000","last_peered":"2024-04-28T12:49:57.193436+0000","last_clean":"2024-04-28T12:49:57.193436+0000","last_became_active":"2024-04-28T12:48:59.192864+0000","last_became_peered":"2024-04-28T12:48:59.192864+0000","last_unstale":"2024-04-28T12:49:57.193436+0000","last_undegraded":"2024-04-28T12:49:57.193436+0000","last_fullsized":"2024-04-28T12:49:57.193436+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:21:18.783650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.212024+0000","last_change":"2024-04-28T12:48:54.132973+0000","last_active":"2024-04-28T12:49:57.212024+0000","last_peered":"2024-04-28T12:49:57.212024+0000","last_clean":"2024-04-28T12:49:57.212024+0000","last_became_active":"2024-04-28T12:48:54.132590+0000","last_became_peered":"2024-04-28T12:48:54.132590+0000","last_unstale":"2024-04-28T12:49:57.212024+0000","last_undegraded":"2024-04-28T12:49:57.212024+0000","last_fullsized":"2024-04-28T12:49:57.212024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:50:52.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.203743+0000","last_change":"2024-04-28T12:48:54.975902+0000","last_active":"2024-04-28T12:49:57.203743+0000","last_peered":"2024-04-28T12:49:57.203743+0000","last_clean":"2024-04-28T12:49:57.203743+0000","last_became_active":"2024-04-28T12:48:54.975488+0000","last_became_peered":"2024-04-28T12:48:54.975488+0000","last_unstale":"2024-04-28T12:49:57.203743+0000","last_undegraded":"2024-04-28T12:49:57.203743+0000","last_fullsized":"2024-04-28T12:49:57.203743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:15:10.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193619+0000","last_change":"2024-04-28T12:48:59.187912+0000","last_active":"2024-04-28T12:49:57.193619+0000","last_peered":"2024-04-28T12:49:57.193619+0000","last_clean":"2024-04-28T12:49:57.193619+0000","last_became_active":"2024-04-28T12:48:59.187427+0000","last_became_peered":"2024-04-28T12:48:59.187427+0000","last_unstale":"2024-04-28T12:49:57.193619+0000","last_undegraded":"2024-04-28T12:49:57.193619+0000","last_fullsized":"2024-04-28T12:49:57.193619+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:27:53.137136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198094+0000","last_change":"2024-04-28T12:48:54.117132+0000","last_active":"2024-04-28T12:49:57.198094+0000","last_peered":"2024-04-28T12:49:57.198094+0000","last_clean":"2024-04-28T12:49:57.198094+0000","last_became_active":"2024-04-28T12:48:54.116805+0000","last_became_peered":"2024-04-28T12:48:54.116805+0000","last_unstale":"2024-04-28T12:49:57.198094+0000","last_undegraded":"2024-04-28T12:49:57.198094+0000","last_fullsized":"2024-04-28T12:49:57.198094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:22:52.015199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199040+0000","last_change":"2024-04-28T12:48:54.972975+0000","last_active":"2024-04-28T12:49:57.199040+0000","last_peered":"2024-04-28T12:49:57.199040+0000","last_clean":"2024-04-28T12:49:57.199040+0000","last_became_active":"2024-04-28T12:48:54.972679+0000","last_became_peered":"2024-04-28T12:48:54.972679+0000","last_unstale":"2024-04-28T12:49:57.199040+0000","last_undegraded":"2024-04-28T12:49:57.199040+0000","last_fullsized":"2024-04-28T12:49:57.199040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:22.032459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193501+0000","last_change":"2024-04-28T12:48:59.187828+0000","last_active":"2024-04-28T12:49:57.193501+0000","last_peered":"2024-04-28T12:49:57.193501+0000","last_clean":"2024-04-28T12:49:57.193501+0000","last_became_active":"2024-04-28T12:48:59.187435+0000","last_became_peered":"2024-04-28T12:48:59.187435+0000","last_unstale":"2024-04-28T12:49:57.193501+0000","last_undegraded":"2024-04-28T12:49:57.193501+0000","last_fullsized":"2024-04-28T12:49:57.193501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:45:22.909378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197990+0000","last_change":"2024-04-28T12:48:54.112455+0000","last_active":"2024-04-28T12:49:57.197990+0000","last_peered":"2024-04-28T12:49:57.197990+0000","last_clean":"2024-04-28T12:49:57.197990+0000","last_became_active":"2024-04-28T12:48:54.111927+0000","last_became_peered":"2024-04-28T12:48:54.111927+0000","last_unstale":"2024-04-28T12:49:57.197990+0000","last_undegraded":"2024-04-28T12:49:57.197990+0000","last_fullsized":"2024-04-28T12:49:57.197990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:32:54.650426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195829+0000","last_change":"2024-04-28T12:48:54.959687+0000","last_active":"2024-04-28T12:49:57.195829+0000","last_peered":"2024-04-28T12:49:57.195829+0000","last_clean":"2024-04-28T12:49:57.195829+0000","last_became_active":"2024-04-28T12:48:54.959355+0000","last_became_peered":"2024-04-28T12:48:54.959355+0000","last_unstale":"2024-04-28T12:49:57.195829+0000","last_undegraded":"2024-04-28T12:49:57.195829+0000","last_fullsized":"2024-04-28T12:49:57.195829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:53:53.919155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193534+0000","last_change":"2024-04-28T12:48:59.174209+0000","last_active":"2024-04-28T12:49:57.193534+0000","last_peered":"2024-04-28T12:49:57.193534+0000","last_clean":"2024-04-28T12:49:57.193534+0000","last_became_active":"2024-04-28T12:48:59.173700+0000","last_became_peered":"2024-04-28T12:48:59.173700+0000","last_unstale":"2024-04-28T12:49:57.193534+0000","last_undegraded":"2024-04-28T12:49:57.193534+0000","last_fullsized":"2024-04-28T12:49:57.193534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:19:11.820482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:47.141864+0000","last_change":"2024-04-28T12:48:54.118061+0000","last_active":"2024-04-28T12:50:47.141864+0000","last_peered":"2024-04-28T12:50:47.141864+0000","last_clean":"2024-04-28T12:50:47.141864+0000","last_became_active":"2024-04-28T12:48:54.117663+0000","last_became_peered":"2024-04-28T12:48:54.117663+0000","last_unstale":"2024-04-28T12:50:47.141864+0000","last_undegraded":"2024-04-28T12:50:47.141864+0000","last_fullsized":"2024-04-28T12:50:47.141864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:41:45.206989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197749+0000","last_change":"2024-04-28T12:48:54.975039+0000","last_active":"2024-04-28T12:49:57.197749+0000","last_peered":"2024-04-28T12:49:57.197749+0000","last_clean":"2024-04-28T12:49:57.197749+0000","last_became_active":"2024-04-28T12:48:54.974589+0000","last_became_peered":"2024-04-28T12:48:54.974589+0000","last_unstale":"2024-04-28T12:49:57.197749+0000","last_undegraded":"2024-04-28T12:49:57.197749+0000","last_fullsized":"2024-04-28T12:49:57.197749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:30.550231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193680+0000","last_change":"2024-04-28T12:48:59.183009+0000","last_active":"2024-04-28T12:49:57.193680+0000","last_peered":"2024-04-28T12:49:57.193680+0000","last_clean":"2024-04-28T12:49:57.193680+0000","last_became_active":"2024-04-28T12:48:59.182641+0000","last_became_peered":"2024-04-28T12:48:59.182641+0000","last_unstale":"2024-04-28T12:49:57.193680+0000","last_undegraded":"2024-04-28T12:49:57.193680+0000","last_fullsized":"2024-04-28T12:49:57.193680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:21.059396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436335+0000","last_change":"2024-04-28T12:48:54.120770+0000","last_active":"2024-04-28T12:49:57.436335+0000","last_peered":"2024-04-28T12:49:57.436335+0000","last_clean":"2024-04-28T12:49:57.436335+0000","last_became_active":"2024-04-28T12:48:54.120429+0000","last_became_peered":"2024-04-28T12:48:54.120429+0000","last_unstale":"2024-04-28T12:49:57.436335+0000","last_undegraded":"2024-04-28T12:49:57.436335+0000","last_fullsized":"2024-04-28T12:49:57.436335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:27:28.172879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195634+0000","last_change":"2024-04-28T12:48:54.982623+0000","last_active":"2024-04-28T12:49:57.195634+0000","last_peered":"2024-04-28T12:49:57.195634+0000","last_clean":"2024-04-28T12:49:57.195634+0000","last_became_active":"2024-04-28T12:48:54.982225+0000","last_became_peered":"2024-04-28T12:48:54.982225+0000","last_unstale":"2024-04-28T12:49:57.195634+0000","last_undegraded":"2024-04-28T12:49:57.195634+0000","last_fullsized":"2024-04-28T12:49:57.195634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:40:17.652783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195291+0000","last_change":"2024-04-28T12:48:59.188489+0000","last_active":"2024-04-28T12:49:57.195291+0000","last_peered":"2024-04-28T12:49:57.195291+0000","last_clean":"2024-04-28T12:49:57.195291+0000","last_became_active":"2024-04-28T12:48:59.188008+0000","last_became_peered":"2024-04-28T12:48:59.188008+0000","last_unstale":"2024-04-28T12:49:57.195291+0000","last_undegraded":"2024-04-28T12:49:57.195291+0000","last_fullsized":"2024-04-28T12:49:57.195291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:34:55.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.206549+0000","last_change":"2024-04-28T12:48:54.137567+0000","last_active":"2024-04-28T12:49:57.206549+0000","last_peered":"2024-04-28T12:49:57.206549+0000","last_clean":"2024-04-28T12:49:57.206549+0000","last_became_active":"2024-04-28T12:48:54.137009+0000","last_became_peered":"2024-04-28T12:48:54.137009+0000","last_unstale":"2024-04-28T12:49:57.206549+0000","last_undegraded":"2024-04-28T12:49:57.206549+0000","last_fullsized":"2024-04-28T12:49:57.206549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:30.909066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.211593+0000","last_change":"2024-04-28T12:48:54.974583+0000","last_active":"2024-04-28T12:49:57.211593+0000","last_peered":"2024-04-28T12:49:57.211593+0000","last_clean":"2024-04-28T12:49:57.211593+0000","last_became_active":"2024-04-28T12:48:54.974217+0000","last_became_peered":"2024-04-28T12:48:54.974217+0000","last_unstale":"2024-04-28T12:49:57.211593+0000","last_undegraded":"2024-04-28T12:49:57.211593+0000","last_fullsized":"2024-04-28T12:49:57.211593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:52:05.768593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.201305+0000","last_change":"2024-04-28T12:48:59.185263+0000","last_active":"2024-04-28T12:49:57.201305+0000","last_peered":"2024-04-28T12:49:57.201305+0000","last_clean":"2024-04-28T12:49:57.201305+0000","last_became_active":"2024-04-28T12:48:59.184423+0000","last_became_peered":"2024-04-28T12:48:59.184423+0000","last_unstale":"2024-04-28T12:49:57.201305+0000","last_undegraded":"2024-04-28T12:49:57.201305+0000","last_fullsized":"2024-04-28T12:49:57.201305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:31:01.029985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195623+0000","last_change":"2024-04-28T12:48:54.119423+0000","last_active":"2024-04-28T12:49:57.195623+0000","last_peered":"2024-04-28T12:49:57.195623+0000","last_clean":"2024-04-28T12:49:57.195623+0000","last_became_active":"2024-04-28T12:48:54.119110+0000","last_became_peered":"2024-04-28T12:48:54.119110+0000","last_unstale":"2024-04-28T12:49:57.195623+0000","last_undegraded":"2024-04-28T12:49:57.195623+0000","last_fullsized":"2024-04-28T12:49:57.195623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:16:33.181397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437777+0000","last_change":"2024-04-28T12:48:54.983978+0000","last_active":"2024-04-28T12:49:57.437777+0000","last_peered":"2024-04-28T12:49:57.437777+0000","last_clean":"2024-04-28T12:49:57.437777+0000","last_became_active":"2024-04-28T12:48:54.982386+0000","last_became_peered":"2024-04-28T12:48:54.982386+0000","last_unstale":"2024-04-28T12:49:57.437777+0000","last_undegraded":"2024-04-28T12:49:57.437777+0000","last_fullsized":"2024-04-28T12:49:57.437777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:23.849403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.208765+0000","last_change":"2024-04-28T12:48:59.187771+0000","last_active":"2024-04-28T12:49:57.208765+0000","last_peered":"2024-04-28T12:49:57.208765+0000","last_clean":"2024-04-28T12:49:57.208765+0000","last_became_active":"2024-04-28T12:48:59.187265+0000","last_became_peered":"2024-04-28T12:48:59.187265+0000","last_unstale":"2024-04-28T12:49:57.208765+0000","last_undegraded":"2024-04-28T12:49:57.208765+0000","last_fullsized":"2024-04-28T12:49:57.208765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:51.211998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":184,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:45.201356+0000","last_change":"2024-04-28T12:48:54.135666+0000","last_active":"2024-04-28T12:50:45.201356+0000","last_peered":"2024-04-28T12:50:45.201356+0000","last_clean":"2024-04-28T12:50:45.201356+0000","last_became_active":"2024-04-28T12:48:54.135373+0000","last_became_peered":"2024-04-28T12:48:54.135373+0000","last_unstale":"2024-04-28T12:50:45.201356+0000","last_undegraded":"2024-04-28T12:50:45.201356+0000","last_fullsized":"2024-04-28T12:50:45.201356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:41:52.455552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":153,"num_read_kb":153,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199231+0000","last_change":"2024-04-28T12:48:54.976474+0000","last_active":"2024-04-28T12:49:57.199231+0000","last_peered":"2024-04-28T12:49:57.199231+0000","last_clean":"2024-04-28T12:49:57.199231+0000","last_became_active":"2024-04-28T12:48:54.976133+0000","last_became_peered":"2024-04-28T12:48:54.976133+0000","last_unstale":"2024-04-28T12:49:57.199231+0000","last_undegraded":"2024-04-28T12:49:57.199231+0000","last_fullsized":"2024-04-28T12:49:57.199231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:24:19.046535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437452+0000","last_change":"2024-04-28T12:48:59.181836+0000","last_active":"2024-04-28T12:49:57.437452+0000","last_peered":"2024-04-28T12:49:57.437452+0000","last_clean":"2024-04-28T12:49:57.437452+0000","last_became_active":"2024-04-28T12:48:59.177499+0000","last_became_peered":"2024-04-28T12:48:59.177499+0000","last_unstale":"2024-04-28T12:49:57.437452+0000","last_undegraded":"2024-04-28T12:49:57.437452+0000","last_fullsized":"2024-04-28T12:49:57.437452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:36:37.981250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198033+0000","last_change":"2024-04-28T12:48:54.118519+0000","last_active":"2024-04-28T12:49:57.198033+0000","last_peered":"2024-04-28T12:49:57.198033+0000","last_clean":"2024-04-28T12:49:57.198033+0000","last_became_active":"2024-04-28T12:48:54.116755+0000","last_became_peered":"2024-04-28T12:48:54.116755+0000","last_unstale":"2024-04-28T12:49:57.198033+0000","last_undegraded":"2024-04-28T12:49:57.198033+0000","last_fullsized":"2024-04-28T12:49:57.198033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:05:46.261098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.210551+0000","last_change":"2024-04-28T12:49:57.210551+0000","last_active":"2024-04-28T12:49:57.210551+0000","last_peered":"2024-04-28T12:49:57.210551+0000","last_clean":"2024-04-28T12:49:57.210551+0000","last_became_active":"2024-04-28T12:49:57.209805+0000","last_became_peered":"2024-04-28T12:49:57.209805+0000","last_unstale":"2024-04-28T12:49:57.210551+0000","last_undegraded":"2024-04-28T12:49:57.210551+0000","last_fullsized":"2024-04-28T12:49:57.210551+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:30:29.206125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193870+0000","last_change":"2024-04-28T12:48:59.179857+0000","last_active":"2024-04-28T12:49:57.193870+0000","last_peered":"2024-04-28T12:49:57.193870+0000","last_clean":"2024-04-28T12:49:57.193870+0000","last_became_active":"2024-04-28T12:48:59.179412+0000","last_became_peered":"2024-04-28T12:48:59.179412+0000","last_unstale":"2024-04-28T12:49:57.193870+0000","last_undegraded":"2024-04-28T12:49:57.193870+0000","last_fullsized":"2024-04-28T12:49:57.193870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:28:08.883832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195630+0000","last_change":"2024-04-28T12:48:54.123771+0000","last_active":"2024-04-28T12:49:57.195630+0000","last_peered":"2024-04-28T12:49:57.195630+0000","last_clean":"2024-04-28T12:49:57.195630+0000","last_became_active":"2024-04-28T12:48:54.123524+0000","last_became_peered":"2024-04-28T12:48:54.123524+0000","last_unstale":"2024-04-28T12:49:57.195630+0000","last_undegraded":"2024-04-28T12:49:57.195630+0000","last_fullsized":"2024-04-28T12:49:57.195630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:59:59.982340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199003+0000","last_change":"2024-04-28T12:48:54.966190+0000","last_active":"2024-04-28T12:49:57.199003+0000","last_peered":"2024-04-28T12:49:57.199003+0000","last_clean":"2024-04-28T12:49:57.199003+0000","last_became_active":"2024-04-28T12:48:54.965854+0000","last_became_peered":"2024-04-28T12:48:54.965854+0000","last_unstale":"2024-04-28T12:49:57.199003+0000","last_undegraded":"2024-04-28T12:49:57.199003+0000","last_fullsized":"2024-04-28T12:49:57.199003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:34:16.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437785+0000","last_change":"2024-04-28T12:48:59.182747+0000","last_active":"2024-04-28T12:49:57.437785+0000","last_peered":"2024-04-28T12:49:57.437785+0000","last_clean":"2024-04-28T12:49:57.437785+0000","last_became_active":"2024-04-28T12:48:59.179610+0000","last_became_peered":"2024-04-28T12:48:59.179610+0000","last_unstale":"2024-04-28T12:49:57.437785+0000","last_undegraded":"2024-04-28T12:49:57.437785+0000","last_fullsized":"2024-04-28T12:49:57.437785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:19:19.603188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'32","reported_seq":219,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:46.139566+0000","last_change":"2024-04-28T12:48:54.116386+0000","last_active":"2024-04-28T12:50:46.139566+0000","last_peered":"2024-04-28T12:50:46.139566+0000","last_clean":"2024-04-28T12:50:46.139566+0000","last_became_active":"2024-04-28T12:48:54.115547+0000","last_became_peered":"2024-04-28T12:48:54.115547+0000","last_unstale":"2024-04-28T12:50:46.139566+0000","last_undegraded":"2024-04-28T12:50:46.139566+0000","last_fullsized":"2024-04-28T12:50:46.139566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:10:17.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51771,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":160,"num_read_kb":130,"num_write":34,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197749+0000","last_change":"2024-04-28T12:48:54.962221+0000","last_active":"2024-04-28T12:49:57.197749+0000","last_peered":"2024-04-28T12:49:57.197749+0000","last_clean":"2024-04-28T12:49:57.197749+0000","last_became_active":"2024-04-28T12:48:54.961876+0000","last_became_peered":"2024-04-28T12:48:54.961876+0000","last_unstale":"2024-04-28T12:49:57.197749+0000","last_undegraded":"2024-04-28T12:49:57.197749+0000","last_fullsized":"2024-04-28T12:49:57.197749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:22:38.067688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.196171+0000","last_change":"2024-04-28T12:48:59.193852+0000","last_active":"2024-04-28T12:49:57.196171+0000","last_peered":"2024-04-28T12:49:57.196171+0000","last_clean":"2024-04-28T12:49:57.196171+0000","last_became_active":"2024-04-28T12:48:59.193100+0000","last_became_peered":"2024-04-28T12:48:59.193100+0000","last_unstale":"2024-04-28T12:49:57.196171+0000","last_undegraded":"2024-04-28T12:49:57.196171+0000","last_fullsized":"2024-04-28T12:49:57.196171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:35:26.732962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193667+0000","last_change":"2024-04-28T12:48:54.129317+0000","last_active":"2024-04-28T12:49:57.193667+0000","last_peered":"2024-04-28T12:49:57.193667+0000","last_clean":"2024-04-28T12:49:57.193667+0000","last_became_active":"2024-04-28T12:48:54.128980+0000","last_became_peered":"2024-04-28T12:48:54.128980+0000","last_unstale":"2024-04-28T12:49:57.193667+0000","last_undegraded":"2024-04-28T12:49:57.193667+0000","last_fullsized":"2024-04-28T12:49:57.193667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:02:34.543478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202921+0000","last_change":"2024-04-28T12:48:54.983576+0000","last_active":"2024-04-28T12:49:57.202921+0000","last_peered":"2024-04-28T12:49:57.202921+0000","last_clean":"2024-04-28T12:49:57.202921+0000","last_became_active":"2024-04-28T12:48:54.983213+0000","last_became_peered":"2024-04-28T12:48:54.983213+0000","last_unstale":"2024-04-28T12:49:57.202921+0000","last_undegraded":"2024-04-28T12:49:57.202921+0000","last_fullsized":"2024-04-28T12:49:57.202921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:22:10.901393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197230+0000","last_change":"2024-04-28T12:48:59.191085+0000","last_active":"2024-04-28T12:49:57.197230+0000","last_peered":"2024-04-28T12:49:57.197230+0000","last_clean":"2024-04-28T12:49:57.197230+0000","last_became_active":"2024-04-28T12:48:59.190431+0000","last_became_peered":"2024-04-28T12:48:59.190431+0000","last_unstale":"2024-04-28T12:49:57.197230+0000","last_undegraded":"2024-04-28T12:49:57.197230+0000","last_fullsized":"2024-04-28T12:49:57.197230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:13:30.990816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.196492+0000","last_change":"2024-04-28T12:48:54.129634+0000","last_active":"2024-04-28T12:49:57.196492+0000","last_peered":"2024-04-28T12:49:57.196492+0000","last_clean":"2024-04-28T12:49:57.196492+0000","last_became_active":"2024-04-28T12:48:54.129250+0000","last_became_peered":"2024-04-28T12:48:54.129250+0000","last_unstale":"2024-04-28T12:49:57.196492+0000","last_undegraded":"2024-04-28T12:49:57.196492+0000","last_fullsized":"2024-04-28T12:49:57.196492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:48:23.828967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197574+0000","last_change":"2024-04-28T12:48:54.978551+0000","last_active":"2024-04-28T12:49:57.197574+0000","last_peered":"2024-04-28T12:49:57.197574+0000","last_clean":"2024-04-28T12:49:57.197574+0000","last_became_active":"2024-04-28T12:48:54.978193+0000","last_became_peered":"2024-04-28T12:48:54.978193+0000","last_unstale":"2024-04-28T12:49:57.197574+0000","last_undegraded":"2024-04-28T12:49:57.197574+0000","last_fullsized":"2024-04-28T12:49:57.197574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:05:31.289174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437926+0000","last_change":"2024-04-28T12:48:59.195185+0000","last_active":"2024-04-28T12:49:57.437926+0000","last_peered":"2024-04-28T12:49:57.437926+0000","last_clean":"2024-04-28T12:49:57.437926+0000","last_became_active":"2024-04-28T12:48:59.194694+0000","last_became_peered":"2024-04-28T12:48:59.194694+0000","last_unstale":"2024-04-28T12:49:57.437926+0000","last_undegraded":"2024-04-28T12:49:57.437926+0000","last_fullsized":"2024-04-28T12:49:57.437926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:20:32.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:47.178087+0000","last_change":"2024-04-28T12:48:54.115249+0000","last_active":"2024-04-28T12:50:47.178087+0000","last_peered":"2024-04-28T12:50:47.178087+0000","last_clean":"2024-04-28T12:50:47.178087+0000","last_became_active":"2024-04-28T12:48:54.114888+0000","last_became_peered":"2024-04-28T12:48:54.114888+0000","last_unstale":"2024-04-28T12:50:47.178087+0000","last_undegraded":"2024-04-28T12:50:47.178087+0000","last_fullsized":"2024-04-28T12:50:47.178087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:30.899021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197425+0000","last_change":"2024-04-28T12:48:54.960038+0000","last_active":"2024-04-28T12:49:57.197425+0000","last_peered":"2024-04-28T12:49:57.197425+0000","last_clean":"2024-04-28T12:49:57.197425+0000","last_became_active":"2024-04-28T12:48:54.959710+0000","last_became_peered":"2024-04-28T12:48:54.959710+0000","last_unstale":"2024-04-28T12:49:57.197425+0000","last_undegraded":"2024-04-28T12:49:57.197425+0000","last_fullsized":"2024-04-28T12:49:57.197425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:13.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.203299+0000","last_change":"2024-04-28T12:48:59.197523+0000","last_active":"2024-04-28T12:49:57.203299+0000","last_peered":"2024-04-28T12:49:57.203299+0000","last_clean":"2024-04-28T12:49:57.203299+0000","last_became_active":"2024-04-28T12:48:59.197088+0000","last_became_peered":"2024-04-28T12:48:59.197088+0000","last_unstale":"2024-04-28T12:49:57.203299+0000","last_undegraded":"2024-04-28T12:49:57.203299+0000","last_fullsized":"2024-04-28T12:49:57.203299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:53:33.391084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197875+0000","last_change":"2024-04-28T12:48:54.130359+0000","last_active":"2024-04-28T12:49:57.197875+0000","last_peered":"2024-04-28T12:49:57.197875+0000","last_clean":"2024-04-28T12:49:57.197875+0000","last_became_active":"2024-04-28T12:48:54.130043+0000","last_became_peered":"2024-04-28T12:48:54.130043+0000","last_unstale":"2024-04-28T12:49:57.197875+0000","last_undegraded":"2024-04-28T12:49:57.197875+0000","last_fullsized":"2024-04-28T12:49:57.197875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:05:41.353359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.200138+0000","last_change":"2024-04-28T12:49:57.200138+0000","last_active":"2024-04-28T12:49:57.200138+0000","last_peered":"2024-04-28T12:49:57.200138+0000","last_clean":"2024-04-28T12:49:57.200138+0000","last_became_active":"2024-04-28T12:49:57.199869+0000","last_became_peered":"2024-04-28T12:49:57.199869+0000","last_unstale":"2024-04-28T12:49:57.200138+0000","last_undegraded":"2024-04-28T12:49:57.200138+0000","last_fullsized":"2024-04-28T12:49:57.200138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:52:01.476631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.196809+0000","last_change":"2024-04-28T12:48:59.193963+0000","last_active":"2024-04-28T12:49:57.196809+0000","last_peered":"2024-04-28T12:49:57.196809+0000","last_clean":"2024-04-28T12:49:57.196809+0000","last_became_active":"2024-04-28T12:48:59.193551+0000","last_became_peered":"2024-04-28T12:48:59.193551+0000","last_unstale":"2024-04-28T12:49:57.196809+0000","last_undegraded":"2024-04-28T12:49:57.196809+0000","last_fullsized":"2024-04-28T12:49:57.196809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:24:29.474296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198030+0000","last_change":"2024-04-28T12:48:54.109277+0000","last_active":"2024-04-28T12:49:57.198030+0000","last_peered":"2024-04-28T12:49:57.198030+0000","last_clean":"2024-04-28T12:49:57.198030+0000","last_became_active":"2024-04-28T12:48:54.108931+0000","last_became_peered":"2024-04-28T12:48:54.108931+0000","last_unstale":"2024-04-28T12:49:57.198030+0000","last_undegraded":"2024-04-28T12:49:57.198030+0000","last_fullsized":"2024-04-28T12:49:57.198030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:56:29.683311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195331+0000","last_change":"2024-04-28T12:48:54.961014+0000","last_active":"2024-04-28T12:49:57.195331+0000","last_peered":"2024-04-28T12:49:57.195331+0000","last_clean":"2024-04-28T12:49:57.195331+0000","last_became_active":"2024-04-28T12:48:54.960648+0000","last_became_peered":"2024-04-28T12:48:54.960648+0000","last_unstale":"2024-04-28T12:49:57.195331+0000","last_undegraded":"2024-04-28T12:49:57.195331+0000","last_fullsized":"2024-04-28T12:49:57.195331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:02:42.794451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197715+0000","last_change":"2024-04-28T12:48:59.185458+0000","last_active":"2024-04-28T12:49:57.197715+0000","last_peered":"2024-04-28T12:49:57.197715+0000","last_clean":"2024-04-28T12:49:57.197715+0000","last_became_active":"2024-04-28T12:48:59.184959+0000","last_became_peered":"2024-04-28T12:48:59.184959+0000","last_unstale":"2024-04-28T12:49:57.197715+0000","last_undegraded":"2024-04-28T12:49:57.197715+0000","last_fullsized":"2024-04-28T12:49:57.197715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:46:45.723374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.438375+0000","last_change":"2024-04-28T12:48:54.134303+0000","last_active":"2024-04-28T12:49:57.438375+0000","last_peered":"2024-04-28T12:49:57.438375+0000","last_clean":"2024-04-28T12:49:57.438375+0000","last_became_active":"2024-04-28T12:48:54.133728+0000","last_became_peered":"2024-04-28T12:48:54.133728+0000","last_unstale":"2024-04-28T12:49:57.438375+0000","last_undegraded":"2024-04-28T12:49:57.438375+0000","last_fullsized":"2024-04-28T12:49:57.438375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:44:16.444168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436554+0000","last_change":"2024-04-28T12:48:54.983312+0000","last_active":"2024-04-28T12:49:57.436554+0000","last_peered":"2024-04-28T12:49:57.436554+0000","last_clean":"2024-04-28T12:49:57.436554+0000","last_became_active":"2024-04-28T12:48:54.982928+0000","last_became_peered":"2024-04-28T12:48:54.982928+0000","last_unstale":"2024-04-28T12:49:57.436554+0000","last_undegraded":"2024-04-28T12:49:57.436554+0000","last_fullsized":"2024-04-28T12:49:57.436554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:18:22.411048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202012+0000","last_change":"2024-04-28T12:48:59.187641+0000","last_active":"2024-04-28T12:49:57.202012+0000","last_peered":"2024-04-28T12:49:57.202012+0000","last_clean":"2024-04-28T12:49:57.202012+0000","last_became_active":"2024-04-28T12:48:59.186003+0000","last_became_peered":"2024-04-28T12:48:59.186003+0000","last_unstale":"2024-04-28T12:49:57.202012+0000","last_undegraded":"2024-04-28T12:49:57.202012+0000","last_fullsized":"2024-04-28T12:49:57.202012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:50:34.968976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":108,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:43.197762+0000","last_change":"2024-04-28T12:48:54.122909+0000","last_active":"2024-04-28T12:50:43.197762+0000","last_peered":"2024-04-28T12:50:43.197762+0000","last_clean":"2024-04-28T12:50:43.197762+0000","last_became_active":"2024-04-28T12:48:54.122661+0000","last_became_peered":"2024-04-28T12:48:54.122661+0000","last_unstale":"2024-04-28T12:50:43.197762+0000","last_undegraded":"2024-04-28T12:50:43.197762+0000","last_fullsized":"2024-04-28T12:50:43.197762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:43:06.119038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436368+0000","last_change":"2024-04-28T12:48:54.956618+0000","last_active":"2024-04-28T12:49:57.436368+0000","last_peered":"2024-04-28T12:49:57.436368+0000","last_clean":"2024-04-28T12:49:57.436368+0000","last_became_active":"2024-04-28T12:48:54.956227+0000","last_became_peered":"2024-04-28T12:48:54.956227+0000","last_unstale":"2024-04-28T12:49:57.436368+0000","last_undegraded":"2024-04-28T12:49:57.436368+0000","last_fullsized":"2024-04-28T12:49:57.436368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:38:02.820771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.203682+0000","last_change":"2024-04-28T12:48:59.187017+0000","last_active":"2024-04-28T12:49:57.203682+0000","last_peered":"2024-04-28T12:49:57.203682+0000","last_clean":"2024-04-28T12:49:57.203682+0000","last_became_active":"2024-04-28T12:48:59.186623+0000","last_became_peered":"2024-04-28T12:48:59.186623+0000","last_unstale":"2024-04-28T12:49:57.203682+0000","last_undegraded":"2024-04-28T12:49:57.203682+0000","last_fullsized":"2024-04-28T12:49:57.203682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:26:53.631400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195818+0000","last_change":"2024-04-28T12:48:54.957348+0000","last_active":"2024-04-28T12:49:57.195818+0000","last_peered":"2024-04-28T12:49:57.195818+0000","last_clean":"2024-04-28T12:49:57.195818+0000","last_became_active":"2024-04-28T12:48:54.956890+0000","last_became_peered":"2024-04-28T12:48:54.956890+0000","last_unstale":"2024-04-28T12:49:57.195818+0000","last_undegraded":"2024-04-28T12:49:57.195818+0000","last_fullsized":"2024-04-28T12:49:57.195818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:41:24.550175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:46.175639+0000","last_change":"2024-04-28T12:48:54.133596+0000","last_active":"2024-04-28T12:50:46.175639+0000","last_peered":"2024-04-28T12:50:46.175639+0000","last_clean":"2024-04-28T12:50:46.175639+0000","last_became_active":"2024-04-28T12:48:54.131632+0000","last_became_peered":"2024-04-28T12:48:54.131632+0000","last_unstale":"2024-04-28T12:50:46.175639+0000","last_undegraded":"2024-04-28T12:50:46.175639+0000","last_fullsized":"2024-04-28T12:50:46.175639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:42:39.352397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197920+0000","last_change":"2024-04-28T12:48:59.174117+0000","last_active":"2024-04-28T12:49:57.197920+0000","last_peered":"2024-04-28T12:49:57.197920+0000","last_clean":"2024-04-28T12:49:57.197920+0000","last_became_active":"2024-04-28T12:48:59.173699+0000","last_became_peered":"2024-04-28T12:48:59.173699+0000","last_unstale":"2024-04-28T12:49:57.197920+0000","last_undegraded":"2024-04-28T12:49:57.197920+0000","last_fullsized":"2024-04-28T12:49:57.197920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:41:51.186413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.211167+0000","last_change":"2024-04-28T12:48:54.978658+0000","last_active":"2024-04-28T12:49:57.211167+0000","last_peered":"2024-04-28T12:49:57.211167+0000","last_clean":"2024-04-28T12:49:57.211167+0000","last_became_active":"2024-04-28T12:48:54.978113+0000","last_became_peered":"2024-04-28T12:48:54.978113+0000","last_unstale":"2024-04-28T12:49:57.211167+0000","last_undegraded":"2024-04-28T12:49:57.211167+0000","last_fullsized":"2024-04-28T12:49:57.211167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:33:50.716489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193767+0000","last_change":"2024-04-28T12:48:54.130301+0000","last_active":"2024-04-28T12:49:57.193767+0000","last_peered":"2024-04-28T12:49:57.193767+0000","last_clean":"2024-04-28T12:49:57.193767+0000","last_became_active":"2024-04-28T12:48:54.130027+0000","last_became_peered":"2024-04-28T12:48:54.130027+0000","last_unstale":"2024-04-28T12:49:57.193767+0000","last_undegraded":"2024-04-28T12:49:57.193767+0000","last_fullsized":"2024-04-28T12:49:57.193767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:30:03.883731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":385,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197343+0000","last_change":"2024-04-28T12:47:34.946674+0000","last_active":"2024-04-28T12:49:57.197343+0000","last_peered":"2024-04-28T12:49:57.197343+0000","last_clean":"2024-04-28T12:49:57.197343+0000","last_became_active":"2024-04-28T12:47:34.946149+0000","last_became_peered":"2024-04-28T12:47:34.946149+0000","last_unstale":"2024-04-28T12:49:57.197343+0000","last_undegraded":"2024-04-28T12:49:57.197343+0000","last_fullsized":"2024-04-28T12:49:57.197343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:43:40.694612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:43:40.694612+0000","last_clean_scrub_stamp":"2024-04-28T12:43:40.694612+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:25:44.776805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195704+0000","last_change":"2024-04-28T12:48:59.190783+0000","last_active":"2024-04-28T12:49:57.195704+0000","last_peered":"2024-04-28T12:49:57.195704+0000","last_clean":"2024-04-28T12:49:57.195704+0000","last_became_active":"2024-04-28T12:48:59.190383+0000","last_became_peered":"2024-04-28T12:48:59.190383+0000","last_unstale":"2024-04-28T12:49:57.195704+0000","last_undegraded":"2024-04-28T12:49:57.195704+0000","last_fullsized":"2024-04-28T12:49:57.195704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:39:41.711227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.439823+0000","last_change":"2024-04-28T12:49:57.439823+0000","last_active":"2024-04-28T12:49:57.439823+0000","last_peered":"2024-04-28T12:49:57.439823+0000","last_clean":"2024-04-28T12:49:57.439823+0000","last_became_active":"2024-04-28T12:49:57.439112+0000","last_became_peered":"2024-04-28T12:49:57.439112+0000","last_unstale":"2024-04-28T12:49:57.439823+0000","last_undegraded":"2024-04-28T12:49:57.439823+0000","last_fullsized":"2024-04-28T12:49:57.439823+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:36:28.295692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197627+0000","last_change":"2024-04-28T12:48:54.116384+0000","last_active":"2024-04-28T12:49:57.197627+0000","last_peered":"2024-04-28T12:49:57.197627+0000","last_clean":"2024-04-28T12:49:57.197627+0000","last_became_active":"2024-04-28T12:48:54.114362+0000","last_became_peered":"2024-04-28T12:48:54.114362+0000","last_unstale":"2024-04-28T12:49:57.197627+0000","last_undegraded":"2024-04-28T12:49:57.197627+0000","last_fullsized":"2024-04-28T12:49:57.197627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:20.406641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195864+0000","last_change":"2024-04-28T12:48:59.181370+0000","last_active":"2024-04-28T12:49:57.195864+0000","last_peered":"2024-04-28T12:49:57.195864+0000","last_clean":"2024-04-28T12:49:57.195864+0000","last_became_active":"2024-04-28T12:48:59.180852+0000","last_became_peered":"2024-04-28T12:48:59.180852+0000","last_unstale":"2024-04-28T12:49:57.195864+0000","last_undegraded":"2024-04-28T12:49:57.195864+0000","last_fullsized":"2024-04-28T12:49:57.195864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:08:55.532273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198025+0000","last_change":"2024-04-28T12:48:54.129792+0000","last_active":"2024-04-28T12:49:57.198025+0000","last_peered":"2024-04-28T12:49:57.198025+0000","last_clean":"2024-04-28T12:49:57.198025+0000","last_became_active":"2024-04-28T12:48:54.129437+0000","last_became_peered":"2024-04-28T12:48:54.129437+0000","last_unstale":"2024-04-28T12:49:57.198025+0000","last_undegraded":"2024-04-28T12:49:57.198025+0000","last_fullsized":"2024-04-28T12:49:57.198025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:22:14.658847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202561+0000","last_change":"2024-04-28T12:48:54.980482+0000","last_active":"2024-04-28T12:49:57.202561+0000","last_peered":"2024-04-28T12:49:57.202561+0000","last_clean":"2024-04-28T12:49:57.202561+0000","last_became_active":"2024-04-28T12:48:54.980022+0000","last_became_peered":"2024-04-28T12:48:54.980022+0000","last_unstale":"2024-04-28T12:49:57.202561+0000","last_undegraded":"2024-04-28T12:49:57.202561+0000","last_fullsized":"2024-04-28T12:49:57.202561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:50:11.573598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.211378+0000","last_change":"2024-04-28T12:48:59.188749+0000","last_active":"2024-04-28T12:49:57.211378+0000","last_peered":"2024-04-28T12:49:57.211378+0000","last_clean":"2024-04-28T12:49:57.211378+0000","last_became_active":"2024-04-28T12:48:59.187735+0000","last_became_peered":"2024-04-28T12:48:59.187735+0000","last_unstale":"2024-04-28T12:49:57.211378+0000","last_undegraded":"2024-04-28T12:49:57.211378+0000","last_fullsized":"2024-04-28T12:49:57.211378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:50:42.948378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437935+0000","last_change":"2024-04-28T12:48:54.132690+0000","last_active":"2024-04-28T12:49:57.437935+0000","last_peered":"2024-04-28T12:49:57.437935+0000","last_clean":"2024-04-28T12:49:57.437935+0000","last_became_active":"2024-04-28T12:48:54.132300+0000","last_became_peered":"2024-04-28T12:48:54.132300+0000","last_unstale":"2024-04-28T12:49:57.437935+0000","last_undegraded":"2024-04-28T12:49:57.437935+0000","last_fullsized":"2024-04-28T12:49:57.437935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:14.194355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.212186+0000","last_change":"2024-04-28T12:48:54.981277+0000","last_active":"2024-04-28T12:49:57.212186+0000","last_peered":"2024-04-28T12:49:57.212186+0000","last_clean":"2024-04-28T12:49:57.212186+0000","last_became_active":"2024-04-28T12:48:54.977366+0000","last_became_peered":"2024-04-28T12:48:54.977366+0000","last_unstale":"2024-04-28T12:49:57.212186+0000","last_undegraded":"2024-04-28T12:49:57.212186+0000","last_fullsized":"2024-04-28T12:49:57.212186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:25.323001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193311+0000","last_change":"2024-04-28T12:48:59.174010+0000","last_active":"2024-04-28T12:49:57.193311+0000","last_peered":"2024-04-28T12:49:57.193311+0000","last_clean":"2024-04-28T12:49:57.193311+0000","last_became_active":"2024-04-28T12:48:59.173370+0000","last_became_peered":"2024-04-28T12:48:59.173370+0000","last_unstale":"2024-04-28T12:49:57.193311+0000","last_undegraded":"2024-04-28T12:49:57.193311+0000","last_fullsized":"2024-04-28T12:49:57.193311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:07:43.258318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193814+0000","last_change":"2024-04-28T12:48:54.127134+0000","last_active":"2024-04-28T12:49:57.193814+0000","last_peered":"2024-04-28T12:49:57.193814+0000","last_clean":"2024-04-28T12:49:57.193814+0000","last_became_active":"2024-04-28T12:48:54.126845+0000","last_became_peered":"2024-04-28T12:48:54.126845+0000","last_unstale":"2024-04-28T12:49:57.193814+0000","last_undegraded":"2024-04-28T12:49:57.193814+0000","last_fullsized":"2024-04-28T12:49:57.193814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:43.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.207587+0000","last_change":"2024-04-28T12:49:57.207587+0000","last_active":"2024-04-28T12:49:57.207587+0000","last_peered":"2024-04-28T12:49:57.207587+0000","last_clean":"2024-04-28T12:49:57.207587+0000","last_became_active":"2024-04-28T12:49:57.206688+0000","last_became_peered":"2024-04-28T12:49:57.206688+0000","last_unstale":"2024-04-28T12:49:57.207587+0000","last_undegraded":"2024-04-28T12:49:57.207587+0000","last_fullsized":"2024-04-28T12:49:57.207587+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:15:27.801898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193881+0000","last_change":"2024-04-28T12:48:59.188456+0000","last_active":"2024-04-28T12:49:57.193881+0000","last_peered":"2024-04-28T12:49:57.193881+0000","last_clean":"2024-04-28T12:49:57.193881+0000","last_became_active":"2024-04-28T12:48:59.187984+0000","last_became_peered":"2024-04-28T12:48:59.187984+0000","last_unstale":"2024-04-28T12:49:57.193881+0000","last_undegraded":"2024-04-28T12:49:57.193881+0000","last_fullsized":"2024-04-28T12:49:57.193881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:38:32.653563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198715+0000","last_change":"2024-04-28T12:48:54.125483+0000","last_active":"2024-04-28T12:49:57.198715+0000","last_peered":"2024-04-28T12:49:57.198715+0000","last_clean":"2024-04-28T12:49:57.198715+0000","last_became_active":"2024-04-28T12:48:54.125261+0000","last_became_peered":"2024-04-28T12:48:54.125261+0000","last_unstale":"2024-04-28T12:49:57.198715+0000","last_undegraded":"2024-04-28T12:49:57.198715+0000","last_fullsized":"2024-04-28T12:49:57.198715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:32:24.986128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197447+0000","last_change":"2024-04-28T12:48:54.960662+0000","last_active":"2024-04-28T12:49:57.197447+0000","last_peered":"2024-04-28T12:49:57.197447+0000","last_clean":"2024-04-28T12:49:57.197447+0000","last_became_active":"2024-04-28T12:48:54.960295+0000","last_became_peered":"2024-04-28T12:48:54.960295+0000","last_unstale":"2024-04-28T12:49:57.197447+0000","last_undegraded":"2024-04-28T12:49:57.197447+0000","last_fullsized":"2024-04-28T12:49:57.197447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:26.190901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.438135+0000","last_change":"2024-04-28T12:48:59.194137+0000","last_active":"2024-04-28T12:49:57.438135+0000","last_peered":"2024-04-28T12:49:57.438135+0000","last_clean":"2024-04-28T12:49:57.438135+0000","last_became_active":"2024-04-28T12:48:59.193740+0000","last_became_peered":"2024-04-28T12:48:59.193740+0000","last_unstale":"2024-04-28T12:49:57.438135+0000","last_undegraded":"2024-04-28T12:49:57.438135+0000","last_fullsized":"2024-04-28T12:49:57.438135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:58:52.962196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.208249+0000","last_change":"2024-04-28T12:48:54.140883+0000","last_active":"2024-04-28T12:49:57.208249+0000","last_peered":"2024-04-28T12:49:57.208249+0000","last_clean":"2024-04-28T12:49:57.208249+0000","last_became_active":"2024-04-28T12:48:54.140548+0000","last_became_peered":"2024-04-28T12:48:54.140548+0000","last_unstale":"2024-04-28T12:49:57.208249+0000","last_undegraded":"2024-04-28T12:49:57.208249+0000","last_fullsized":"2024-04-28T12:49:57.208249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:31.465266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195119+0000","last_change":"2024-04-28T12:48:54.976359+0000","last_active":"2024-04-28T12:49:57.195119+0000","last_peered":"2024-04-28T12:49:57.195119+0000","last_clean":"2024-04-28T12:49:57.195119+0000","last_became_active":"2024-04-28T12:48:54.976028+0000","last_became_peered":"2024-04-28T12:48:54.976028+0000","last_unstale":"2024-04-28T12:49:57.195119+0000","last_undegraded":"2024-04-28T12:49:57.195119+0000","last_fullsized":"2024-04-28T12:49:57.195119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:02:48.498276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193667+0000","last_change":"2024-04-28T12:48:59.179755+0000","last_active":"2024-04-28T12:49:57.193667+0000","last_peered":"2024-04-28T12:49:57.193667+0000","last_clean":"2024-04-28T12:49:57.193667+0000","last_became_active":"2024-04-28T12:48:59.179258+0000","last_became_peered":"2024-04-28T12:48:59.179258+0000","last_unstale":"2024-04-28T12:49:57.193667+0000","last_undegraded":"2024-04-28T12:49:57.193667+0000","last_fullsized":"2024-04-28T12:49:57.193667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:59:38.863875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.217857+0000","last_change":"2024-04-28T12:49:57.217468+0000","last_active":"2024-04-28T12:49:57.217857+0000","last_peered":"2024-04-28T12:49:57.217857+0000","last_clean":"2024-04-28T12:49:57.217857+0000","last_became_active":"2024-04-28T12:49:57.212697+0000","last_became_peered":"2024-04-28T12:49:57.212697+0000","last_unstale":"2024-04-28T12:49:57.217857+0000","last_undegraded":"2024-04-28T12:49:57.217857+0000","last_fullsized":"2024-04-28T12:49:57.217857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:57:20.543124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.207772+0000","last_change":"2024-04-28T12:48:54.990809+0000","last_active":"2024-04-28T12:49:57.207772+0000","last_peered":"2024-04-28T12:49:57.207772+0000","last_clean":"2024-04-28T12:49:57.207772+0000","last_became_active":"2024-04-28T12:48:54.990251+0000","last_became_peered":"2024-04-28T12:48:54.990251+0000","last_unstale":"2024-04-28T12:49:57.207772+0000","last_undegraded":"2024-04-28T12:49:57.207772+0000","last_fullsized":"2024-04-28T12:49:57.207772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:14:30.017815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193974+0000","last_change":"2024-04-28T12:48:59.193287+0000","last_active":"2024-04-28T12:49:57.193974+0000","last_peered":"2024-04-28T12:49:57.193974+0000","last_clean":"2024-04-28T12:49:57.193974+0000","last_became_active":"2024-04-28T12:48:59.192755+0000","last_became_peered":"2024-04-28T12:48:59.192755+0000","last_unstale":"2024-04-28T12:49:57.193974+0000","last_undegraded":"2024-04-28T12:49:57.193974+0000","last_fullsized":"2024-04-28T12:49:57.193974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:00:33.790384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.204891+0000","last_change":"2024-04-28T12:49:57.204548+0000","last_active":"2024-04-28T12:49:57.204891+0000","last_peered":"2024-04-28T12:49:57.204891+0000","last_clean":"2024-04-28T12:49:57.204891+0000","last_became_active":"2024-04-28T12:49:57.199078+0000","last_became_peered":"2024-04-28T12:49:57.199078+0000","last_unstale":"2024-04-28T12:49:57.204891+0000","last_undegraded":"2024-04-28T12:49:57.204891+0000","last_fullsized":"2024-04-28T12:49:57.204891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:21:46.153453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197870+0000","last_change":"2024-04-28T12:48:54.970270+0000","last_active":"2024-04-28T12:49:57.197870+0000","last_peered":"2024-04-28T12:49:57.197870+0000","last_clean":"2024-04-28T12:49:57.197870+0000","last_became_active":"2024-04-28T12:48:54.970010+0000","last_became_peered":"2024-04-28T12:48:54.970010+0000","last_unstale":"2024-04-28T12:49:57.197870+0000","last_undegraded":"2024-04-28T12:49:57.197870+0000","last_fullsized":"2024-04-28T12:49:57.197870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:05:03.578751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198646+0000","last_change":"2024-04-28T12:48:59.186197+0000","last_active":"2024-04-28T12:49:57.198646+0000","last_peered":"2024-04-28T12:49:57.198646+0000","last_clean":"2024-04-28T12:49:57.198646+0000","last_became_active":"2024-04-28T12:48:59.185749+0000","last_became_peered":"2024-04-28T12:48:59.185749+0000","last_unstale":"2024-04-28T12:49:57.198646+0000","last_undegraded":"2024-04-28T12:49:57.198646+0000","last_fullsized":"2024-04-28T12:49:57.198646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:36.336775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:44.200422+0000","last_change":"2024-04-28T12:48:54.118725+0000","last_active":"2024-04-28T12:50:44.200422+0000","last_peered":"2024-04-28T12:50:44.200422+0000","last_clean":"2024-04-28T12:50:44.200422+0000","last_became_active":"2024-04-28T12:48:54.118419+0000","last_became_peered":"2024-04-28T12:48:54.118419+0000","last_unstale":"2024-04-28T12:50:44.200422+0000","last_undegraded":"2024-04-28T12:50:44.200422+0000","last_fullsized":"2024-04-28T12:50:44.200422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:25:17.959905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198356+0000","last_change":"2024-04-28T12:48:54.959518+0000","last_active":"2024-04-28T12:49:57.198356+0000","last_peered":"2024-04-28T12:49:57.198356+0000","last_clean":"2024-04-28T12:49:57.198356+0000","last_became_active":"2024-04-28T12:48:54.959101+0000","last_became_peered":"2024-04-28T12:48:54.959101+0000","last_unstale":"2024-04-28T12:49:57.198356+0000","last_undegraded":"2024-04-28T12:49:57.198356+0000","last_fullsized":"2024-04-28T12:49:57.198356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:38:50.496961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199527+0000","last_change":"2024-04-28T12:48:59.180941+0000","last_active":"2024-04-28T12:49:57.199527+0000","last_peered":"2024-04-28T12:49:57.199527+0000","last_clean":"2024-04-28T12:49:57.199527+0000","last_became_active":"2024-04-28T12:48:59.176889+0000","last_became_peered":"2024-04-28T12:48:59.176889+0000","last_unstale":"2024-04-28T12:49:57.199527+0000","last_undegraded":"2024-04-28T12:49:57.199527+0000","last_fullsized":"2024-04-28T12:49:57.199527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:28:21.853493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195932+0000","last_change":"2024-04-28T12:48:54.126265+0000","last_active":"2024-04-28T12:49:57.195932+0000","last_peered":"2024-04-28T12:49:57.195932+0000","last_clean":"2024-04-28T12:49:57.195932+0000","last_became_active":"2024-04-28T12:48:54.125918+0000","last_became_peered":"2024-04-28T12:48:54.125918+0000","last_unstale":"2024-04-28T12:49:57.195932+0000","last_undegraded":"2024-04-28T12:49:57.195932+0000","last_fullsized":"2024-04-28T12:49:57.195932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:24:26.340143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197820+0000","last_change":"2024-04-28T12:48:54.985335+0000","last_active":"2024-04-28T12:49:57.197820+0000","last_peered":"2024-04-28T12:49:57.197820+0000","last_clean":"2024-04-28T12:49:57.197820+0000","last_became_active":"2024-04-28T12:48:54.985017+0000","last_became_peered":"2024-04-28T12:48:54.985017+0000","last_unstale":"2024-04-28T12:49:57.197820+0000","last_undegraded":"2024-04-28T12:49:57.197820+0000","last_fullsized":"2024-04-28T12:49:57.197820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:49:21.821976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195330+0000","last_change":"2024-04-28T12:48:59.176438+0000","last_active":"2024-04-28T12:49:57.195330+0000","last_peered":"2024-04-28T12:49:57.195330+0000","last_clean":"2024-04-28T12:49:57.195330+0000","last_became_active":"2024-04-28T12:48:59.175773+0000","last_became_peered":"2024-04-28T12:48:59.175773+0000","last_unstale":"2024-04-28T12:49:57.195330+0000","last_undegraded":"2024-04-28T12:49:57.195330+0000","last_fullsized":"2024-04-28T12:49:57.195330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:32.687553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202953+0000","last_change":"2024-04-28T12:48:54.130452+0000","last_active":"2024-04-28T12:49:57.202953+0000","last_peered":"2024-04-28T12:49:57.202953+0000","last_clean":"2024-04-28T12:49:57.202953+0000","last_became_active":"2024-04-28T12:48:54.130090+0000","last_became_peered":"2024-04-28T12:48:54.130090+0000","last_unstale":"2024-04-28T12:49:57.202953+0000","last_undegraded":"2024-04-28T12:49:57.202953+0000","last_fullsized":"2024-04-28T12:49:57.202953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:02:34.014710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198233+0000","last_change":"2024-04-28T12:48:54.961838+0000","last_active":"2024-04-28T12:49:57.198233+0000","last_peered":"2024-04-28T12:49:57.198233+0000","last_clean":"2024-04-28T12:49:57.198233+0000","last_became_active":"2024-04-28T12:48:54.961525+0000","last_became_peered":"2024-04-28T12:48:54.961525+0000","last_unstale":"2024-04-28T12:49:57.198233+0000","last_undegraded":"2024-04-28T12:49:57.198233+0000","last_fullsized":"2024-04-28T12:49:57.198233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:06:46.077023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195168+0000","last_change":"2024-04-28T12:48:54.959806+0000","last_active":"2024-04-28T12:49:57.195168+0000","last_peered":"2024-04-28T12:49:57.195168+0000","last_clean":"2024-04-28T12:49:57.195168+0000","last_became_active":"2024-04-28T12:48:54.959491+0000","last_became_peered":"2024-04-28T12:48:54.959491+0000","last_unstale":"2024-04-28T12:49:57.195168+0000","last_undegraded":"2024-04-28T12:49:57.195168+0000","last_fullsized":"2024-04-28T12:49:57.195168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:12:25.367185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193620+0000","last_change":"2024-04-28T12:48:54.960035+0000","last_active":"2024-04-28T12:49:57.193620+0000","last_peered":"2024-04-28T12:49:57.193620+0000","last_clean":"2024-04-28T12:49:57.193620+0000","last_became_active":"2024-04-28T12:48:54.959684+0000","last_became_peered":"2024-04-28T12:48:54.959684+0000","last_unstale":"2024-04-28T12:49:57.193620+0000","last_undegraded":"2024-04-28T12:49:57.193620+0000","last_fullsized":"2024-04-28T12:49:57.193620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:46:36.484928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193393+0000","last_change":"2024-04-28T12:48:54.984682+0000","last_active":"2024-04-28T12:49:57.193393+0000","last_peered":"2024-04-28T12:49:57.193393+0000","last_clean":"2024-04-28T12:49:57.193393+0000","last_became_active":"2024-04-28T12:48:54.984346+0000","last_became_peered":"2024-04-28T12:48:54.984346+0000","last_unstale":"2024-04-28T12:49:57.193393+0000","last_undegraded":"2024-04-28T12:49:57.193393+0000","last_fullsized":"2024-04-28T12:49:57.193393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:22:57.828641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.200399+0000","last_change":"2024-04-28T12:48:54.981526+0000","last_active":"2024-04-28T12:49:57.200399+0000","last_peered":"2024-04-28T12:49:57.200399+0000","last_clean":"2024-04-28T12:49:57.200399+0000","last_became_active":"2024-04-28T12:48:54.981212+0000","last_became_peered":"2024-04-28T12:48:54.981212+0000","last_unstale":"2024-04-28T12:49:57.200399+0000","last_undegraded":"2024-04-28T12:49:57.200399+0000","last_fullsized":"2024-04-28T12:49:57.200399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:29:00.976903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.439882+0000","last_change":"2024-04-28T12:49:57.439882+0000","last_active":"2024-04-28T12:49:57.439882+0000","last_peered":"2024-04-28T12:49:57.439882+0000","last_clean":"2024-04-28T12:49:57.439882+0000","last_became_active":"2024-04-28T12:49:57.439390+0000","last_became_peered":"2024-04-28T12:49:57.439390+0000","last_unstale":"2024-04-28T12:49:57.439882+0000","last_undegraded":"2024-04-28T12:49:57.439882+0000","last_fullsized":"2024-04-28T12:49:57.439882+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:39:01.414195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197213+0000","last_change":"2024-04-28T12:48:54.964342+0000","last_active":"2024-04-28T12:49:57.197213+0000","last_peered":"2024-04-28T12:49:57.197213+0000","last_clean":"2024-04-28T12:49:57.197213+0000","last_became_active":"2024-04-28T12:48:54.963980+0000","last_became_peered":"2024-04-28T12:48:54.963980+0000","last_unstale":"2024-04-28T12:49:57.197213+0000","last_undegraded":"2024-04-28T12:49:57.197213+0000","last_fullsized":"2024-04-28T12:49:57.197213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:17:07.772838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193856+0000","last_change":"2024-04-28T12:48:54.963955+0000","last_active":"2024-04-28T12:49:57.193856+0000","last_peered":"2024-04-28T12:49:57.193856+0000","last_clean":"2024-04-28T12:49:57.193856+0000","last_became_active":"2024-04-28T12:48:54.963572+0000","last_became_peered":"2024-04-28T12:48:54.963572+0000","last_unstale":"2024-04-28T12:49:57.193856+0000","last_undegraded":"2024-04-28T12:49:57.193856+0000","last_fullsized":"2024-04-28T12:49:57.193856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:18.077455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.212365+0000","last_change":"2024-04-28T12:48:54.978143+0000","last_active":"2024-04-28T12:49:57.212365+0000","last_peered":"2024-04-28T12:49:57.212365+0000","last_clean":"2024-04-28T12:49:57.212365+0000","last_became_active":"2024-04-28T12:48:54.977715+0000","last_became_peered":"2024-04-28T12:48:54.977715+0000","last_unstale":"2024-04-28T12:49:57.212365+0000","last_undegraded":"2024-04-28T12:49:57.212365+0000","last_fullsized":"2024-04-28T12:49:57.212365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:47:17.071081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195728+0000","last_change":"2024-04-28T12:48:54.982613+0000","last_active":"2024-04-28T12:49:57.195728+0000","last_peered":"2024-04-28T12:49:57.195728+0000","last_clean":"2024-04-28T12:49:57.195728+0000","last_became_active":"2024-04-28T12:48:54.982231+0000","last_became_peered":"2024-04-28T12:48:54.982231+0000","last_unstale":"2024-04-28T12:49:57.195728+0000","last_undegraded":"2024-04-28T12:49:57.195728+0000","last_fullsized":"2024-04-28T12:49:57.195728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:22.206377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.208447+0000","last_change":"2024-04-28T12:48:54.978398+0000","last_active":"2024-04-28T12:49:57.208447+0000","last_peered":"2024-04-28T12:49:57.208447+0000","last_clean":"2024-04-28T12:49:57.208447+0000","last_became_active":"2024-04-28T12:48:54.977015+0000","last_became_peered":"2024-04-28T12:48:54.977015+0000","last_unstale":"2024-04-28T12:49:57.208447+0000","last_undegraded":"2024-04-28T12:49:57.208447+0000","last_fullsized":"2024-04-28T12:49:57.208447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:39:39.499850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198856+0000","last_change":"2024-04-28T12:48:54.982870+0000","last_active":"2024-04-28T12:49:57.198856+0000","last_peered":"2024-04-28T12:49:57.198856+0000","last_clean":"2024-04-28T12:49:57.198856+0000","last_became_active":"2024-04-28T12:48:54.982308+0000","last_became_peered":"2024-04-28T12:48:54.982308+0000","last_unstale":"2024-04-28T12:49:57.198856+0000","last_undegraded":"2024-04-28T12:49:57.198856+0000","last_fullsized":"2024-04-28T12:49:57.198856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:47:45.976825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202230+0000","last_change":"2024-04-28T12:48:54.987904+0000","last_active":"2024-04-28T12:49:57.202230+0000","last_peered":"2024-04-28T12:49:57.202230+0000","last_clean":"2024-04-28T12:49:57.202230+0000","last_became_active":"2024-04-28T12:48:54.987551+0000","last_became_peered":"2024-04-28T12:48:54.987551+0000","last_unstale":"2024-04-28T12:49:57.202230+0000","last_undegraded":"2024-04-28T12:49:57.202230+0000","last_fullsized":"2024-04-28T12:49:57.202230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:30:36.831005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197169+0000","last_change":"2024-04-28T12:48:54.963176+0000","last_active":"2024-04-28T12:49:57.197169+0000","last_peered":"2024-04-28T12:49:57.197169+0000","last_clean":"2024-04-28T12:49:57.197169+0000","last_became_active":"2024-04-28T12:48:54.962830+0000","last_became_peered":"2024-04-28T12:48:54.962830+0000","last_unstale":"2024-04-28T12:49:57.197169+0000","last_undegraded":"2024-04-28T12:49:57.197169+0000","last_fullsized":"2024-04-28T12:49:57.197169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:10:40.671376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59366,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":807,"num_read_kb":468,"num_write":242,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":178098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":501712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645353,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1140299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808879,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":505347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":501629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332156,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1098517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":496755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":549556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":498602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1110016,"data_stored":550558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":496463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":501220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1658880,"data_stored":1093708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:50:51.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-28T12:50:51.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:51 smithi106 ceph-mon[28081]: pgmap v378: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:50:51.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:51 smithi106 ceph-mon[28081]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:50:51.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:51 smithi171 ceph-mon[63127]: pgmap v378: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:50:51.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:51 smithi171 ceph-mon[63127]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:50:51.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:51 smithi129 ceph-mon[62418]: pgmap v378: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:50:51.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:51 smithi129 ceph-mon[62418]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:50:51.582 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:53.376 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:53 smithi106 ceph-mon[28081]: pgmap v379: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:53.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:53 smithi171 ceph-mon[63127]: pgmap v379: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:53.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:53 smithi129 ceph-mon[62418]: pgmap v379: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:50:53.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.673+0000 7ff5c2400700 1 -- 172.21.15.106:0/1817440300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 msgr2=0x7ff5bc103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:53.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.673+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1817440300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 0x7ff5bc103930 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80099c0 tx=0x7ff5a8009cd0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.673+0000 7ff5c2400700 1 -- 172.21.15.106:0/1817440300 shutdown_connections 2024-04-28T12:50:53.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.673+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1817440300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5bc103e70 0x7ff5bc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.673+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1817440300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5bc109550 0x7ff5bc109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.673+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1817440300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 0x7ff5bc103930 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.673+0000 7ff5c2400700 1 -- 172.21.15.106:0/1817440300 >> 172.21.15.106:0/1817440300 conn(0x7ff5bc07a610 msgr2=0x7ff5bc07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:53.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.673+0000 7ff5c2400700 1 -- 172.21.15.106:0/1817440300 shutdown_connections 2024-04-28T12:50:53.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.673+0000 7ff5c2400700 1 -- 172.21.15.106:0/1817440300 wait complete. 2024-04-28T12:50:53.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.675+0000 7ff5c2400700 1 Processor -- start 2024-04-28T12:50:53.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.675+0000 7ff5c2400700 1 -- start start 2024-04-28T12:50:53.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.675+0000 7ff5c2400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 0x7ff5bc19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:53.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.675+0000 7ff5c2400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5bc103e70 0x7ff5bc19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:53.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5c2400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5bc109550 0x7ff5bc1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:53.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5c2400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc1136a0 con 0x7ff5bc1034b0 2024-04-28T12:50:53.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5c2400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc113520 con 0x7ff5bc109550 2024-04-28T12:50:53.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5c2400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc113820 con 0x7ff5bc103e70 2024-04-28T12:50:53.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bb400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 0x7ff5bc19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:53.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bb400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 0x7ff5bc19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52958/0 (socket says 172.21.15.106:52958) 2024-04-28T12:50:53.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bb400700 1 -- 172.21.15.106:0/1875318740 learned_addr learned my addr 172.21.15.106:0/1875318740 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:53.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bbe00700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5bc109550 0x7ff5bc1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:53.673 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5baa00700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5bc103e70 0x7ff5bc19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:53.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bb400700 1 -- 172.21.15.106:0/1875318740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5bc103e70 msgr2=0x7ff5bc19c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:53.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bb400700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5bc103e70 0x7ff5bc19c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bb400700 1 -- 172.21.15.106:0/1875318740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5bc109550 msgr2=0x7ff5bc1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:53.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bb400700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5bc109550 0x7ff5bc1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bb400700 1 -- 172.21.15.106:0/1875318740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc1a0d60 con 0x7ff5bc1034b0 2024-04-28T12:50:53.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.676+0000 7ff5bbe00700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5bc109550 0x7ff5bc1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:53.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.677+0000 7ff5bb400700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 0x7ff5bc19bd50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00e9c0 tx=0x7ff5ac00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:53.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.677+0000 7ff5b3e00700 1 -- 172.21.15.106:0/1875318740 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac00cbd0 con 0x7ff5bc1034b0 2024-04-28T12:50:53.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.677+0000 7ff5b3e00700 1 -- 172.21.15.106:0/1875318740 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5ac00ee60 con 0x7ff5bc1034b0 2024-04-28T12:50:53.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.677+0000 7ff5b3e00700 1 -- 172.21.15.106:0/1875318740 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0055c0 con 0x7ff5bc1034b0 2024-04-28T12:50:53.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.677+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc1a1050 con 0x7ff5bc1034b0 2024-04-28T12:50:53.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.677+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc1a88d0 con 0x7ff5bc1034b0 2024-04-28T12:50:53.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.678+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc06a110 con 0x7ff5bc1034b0 2024-04-28T12:50:53.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.680+0000 7ff5b3e00700 1 -- 172.21.15.106:0/1875318740 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff5ac004750 con 0x7ff5bc1034b0 2024-04-28T12:50:53.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.682+0000 7ff5b3e00700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5a00a3d50 0x7ff5a00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:53.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.682+0000 7ff5b3e00700 1 -- 172.21.15.106:0/1875318740 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9071+0+0 (secure 0 0 0) 0x7ff5ac0d0b10 con 0x7ff5bc1034b0 2024-04-28T12:50:53.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.682+0000 7ff5baa00700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5a00a3d50 0x7ff5a00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:53.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.683+0000 7ff5baa00700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5a00a3d50 0x7ff5a00a6210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc07af10 tx=0x7ff5a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:53.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.687+0000 7ff5b3e00700 1 -- 172.21.15.106:0/1875318740 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff5ac09cff0 con 0x7ff5bc1034b0 2024-04-28T12:50:53.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.908+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5bc110090 con 0x7ff5a00a3d50 2024-04-28T12:50:53.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.925+0000 7ff5b3e00700 1 -- 172.21.15.106:0/1875318740 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340414 (secure 0 0 0) 0x7ff5bc110090 con 0x7ff5a00a3d50 2024-04-28T12:50:53.929 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:50:53.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5a00a3d50 msgr2=0x7ff5a00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:53.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5a00a3d50 0x7ff5a00a6210 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc07af10 tx=0x7ff5a8005fb0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 msgr2=0x7ff5bc19bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:53.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 0x7ff5bc19bd50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00e9c0 tx=0x7ff5ac00ecd0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 shutdown_connections 2024-04-28T12:50:53.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5bc109550 0x7ff5bc1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5bc1034b0 0x7ff5bc19bd50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff5a00a3d50 0x7ff5a00a6210 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 --2- 172.21.15.106:0/1875318740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5bc103e70 0x7ff5bc19c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:53.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 >> 172.21.15.106:0/1875318740 conn(0x7ff5bc07a610 msgr2=0x7ff5bc10e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:53.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 shutdown_connections 2024-04-28T12:50:53.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:53.934+0000 7ff5c2400700 1 -- 172.21.15.106:0/1875318740 wait complete. 2024-04-28T12:50:53.932 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2024-04-28T12:50:54.729 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-04-28T12:50:52.104614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649968,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":963,"num_read_kb":592,"num_write":425,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":367600,"kb_used_data":14204,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124574224,"statfs":{"total":1151940427776,"available":1151564005376,"internally_reserved":0,"allocated":14544896,"data_stored":7934366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":361674044},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006699"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195899+0000","last_change":"2024-04-28T12:48:54.979081+0000","last_active":"2024-04-28T12:49:57.195899+0000","last_peered":"2024-04-28T12:49:57.195899+0000","last_clean":"2024-04-28T12:49:57.195899+0000","last_became_active":"2024-04-28T12:48:54.978668+0000","last_became_peered":"2024-04-28T12:48:54.978668+0000","last_unstale":"2024-04-28T12:49:57.195899+0000","last_undegraded":"2024-04-28T12:49:57.195899+0000","last_fullsized":"2024-04-28T12:49:57.195899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:33:43.345706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.201410+0000","last_change":"2024-04-28T12:48:54.966277+0000","last_active":"2024-04-28T12:49:57.201410+0000","last_peered":"2024-04-28T12:49:57.201410+0000","last_clean":"2024-04-28T12:49:57.201410+0000","last_became_active":"2024-04-28T12:48:54.965880+0000","last_became_peered":"2024-04-28T12:48:54.965880+0000","last_unstale":"2024-04-28T12:49:57.201410+0000","last_undegraded":"2024-04-28T12:49:57.201410+0000","last_fullsized":"2024-04-28T12:49:57.201410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:42:13.166459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.201177+0000","last_change":"2024-04-28T12:48:54.966516+0000","last_active":"2024-04-28T12:49:57.201177+0000","last_peered":"2024-04-28T12:49:57.201177+0000","last_clean":"2024-04-28T12:49:57.201177+0000","last_became_active":"2024-04-28T12:48:54.966112+0000","last_became_peered":"2024-04-28T12:48:54.966112+0000","last_unstale":"2024-04-28T12:49:57.201177+0000","last_undegraded":"2024-04-28T12:49:57.201177+0000","last_fullsized":"2024-04-28T12:49:57.201177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:15:05.538389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193943+0000","last_change":"2024-04-28T12:48:54.962718+0000","last_active":"2024-04-28T12:49:57.193943+0000","last_peered":"2024-04-28T12:49:57.193943+0000","last_clean":"2024-04-28T12:49:57.193943+0000","last_became_active":"2024-04-28T12:48:54.962371+0000","last_became_peered":"2024-04-28T12:48:54.962371+0000","last_unstale":"2024-04-28T12:49:57.193943+0000","last_undegraded":"2024-04-28T12:49:57.193943+0000","last_fullsized":"2024-04-28T12:49:57.193943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:55:47.714734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197823+0000","last_change":"2024-04-28T12:48:54.986898+0000","last_active":"2024-04-28T12:49:57.197823+0000","last_peered":"2024-04-28T12:49:57.197823+0000","last_clean":"2024-04-28T12:49:57.197823+0000","last_became_active":"2024-04-28T12:48:54.986550+0000","last_became_peered":"2024-04-28T12:48:54.986550+0000","last_unstale":"2024-04-28T12:49:57.197823+0000","last_undegraded":"2024-04-28T12:49:57.197823+0000","last_fullsized":"2024-04-28T12:49:57.197823+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:54:54.942449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436383+0000","last_change":"2024-04-28T12:48:54.981357+0000","last_active":"2024-04-28T12:49:57.436383+0000","last_peered":"2024-04-28T12:49:57.436383+0000","last_clean":"2024-04-28T12:49:57.436383+0000","last_became_active":"2024-04-28T12:48:54.980945+0000","last_became_peered":"2024-04-28T12:48:54.980945+0000","last_unstale":"2024-04-28T12:49:57.436383+0000","last_undegraded":"2024-04-28T12:49:57.436383+0000","last_fullsized":"2024-04-28T12:49:57.436383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:21:51.476872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.439357+0000","last_change":"2024-04-28T12:49:57.439357+0000","last_active":"2024-04-28T12:49:57.439357+0000","last_peered":"2024-04-28T12:49:57.439357+0000","last_clean":"2024-04-28T12:49:57.439357+0000","last_became_active":"2024-04-28T12:49:57.438909+0000","last_became_peered":"2024-04-28T12:49:57.438909+0000","last_unstale":"2024-04-28T12:49:57.439357+0000","last_undegraded":"2024-04-28T12:49:57.439357+0000","last_fullsized":"2024-04-28T12:49:57.439357+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:34:45.679426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.211795+0000","last_change":"2024-04-28T12:48:54.979500+0000","last_active":"2024-04-28T12:49:57.211795+0000","last_peered":"2024-04-28T12:49:57.211795+0000","last_clean":"2024-04-28T12:49:57.211795+0000","last_became_active":"2024-04-28T12:48:54.978541+0000","last_became_peered":"2024-04-28T12:48:54.978541+0000","last_unstale":"2024-04-28T12:49:57.211795+0000","last_undegraded":"2024-04-28T12:49:57.211795+0000","last_fullsized":"2024-04-28T12:49:57.211795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:06:47.550461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.210318+0000","last_change":"2024-04-28T12:49:57.210318+0000","last_active":"2024-04-28T12:49:57.210318+0000","last_peered":"2024-04-28T12:49:57.210318+0000","last_clean":"2024-04-28T12:49:57.210318+0000","last_became_active":"2024-04-28T12:49:57.209987+0000","last_became_peered":"2024-04-28T12:49:57.209987+0000","last_unstale":"2024-04-28T12:49:57.210318+0000","last_undegraded":"2024-04-28T12:49:57.210318+0000","last_fullsized":"2024-04-28T12:49:57.210318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:08:27.279256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,11],"acting":[1,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436725+0000","last_change":"2024-04-28T12:48:54.961425+0000","last_active":"2024-04-28T12:49:57.436725+0000","last_peered":"2024-04-28T12:49:57.436725+0000","last_clean":"2024-04-28T12:49:57.436725+0000","last_became_active":"2024-04-28T12:48:54.961060+0000","last_became_peered":"2024-04-28T12:48:54.961060+0000","last_unstale":"2024-04-28T12:49:57.436725+0000","last_undegraded":"2024-04-28T12:49:57.436725+0000","last_fullsized":"2024-04-28T12:49:57.436725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:49:14.184108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198106+0000","last_change":"2024-04-28T12:48:54.963483+0000","last_active":"2024-04-28T12:49:57.198106+0000","last_peered":"2024-04-28T12:49:57.198106+0000","last_clean":"2024-04-28T12:49:57.198106+0000","last_became_active":"2024-04-28T12:48:54.963146+0000","last_became_peered":"2024-04-28T12:48:54.963146+0000","last_unstale":"2024-04-28T12:49:57.198106+0000","last_undegraded":"2024-04-28T12:49:57.198106+0000","last_fullsized":"2024-04-28T12:49:57.198106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:05:30.313659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193857+0000","last_change":"2024-04-28T12:48:54.961834+0000","last_active":"2024-04-28T12:49:57.193857+0000","last_peered":"2024-04-28T12:49:57.193857+0000","last_clean":"2024-04-28T12:49:57.193857+0000","last_became_active":"2024-04-28T12:48:54.961521+0000","last_became_peered":"2024-04-28T12:48:54.961521+0000","last_unstale":"2024-04-28T12:49:57.193857+0000","last_undegraded":"2024-04-28T12:49:57.193857+0000","last_fullsized":"2024-04-28T12:49:57.193857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:34:57.168391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202673+0000","last_change":"2024-04-28T12:48:54.979673+0000","last_active":"2024-04-28T12:49:57.202673+0000","last_peered":"2024-04-28T12:49:57.202673+0000","last_clean":"2024-04-28T12:49:57.202673+0000","last_became_active":"2024-04-28T12:48:54.976440+0000","last_became_peered":"2024-04-28T12:48:54.976440+0000","last_unstale":"2024-04-28T12:49:57.202673+0000","last_undegraded":"2024-04-28T12:49:57.202673+0000","last_fullsized":"2024-04-28T12:49:57.202673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:46:30.391135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199191+0000","last_change":"2024-04-28T12:48:54.963108+0000","last_active":"2024-04-28T12:49:57.199191+0000","last_peered":"2024-04-28T12:49:57.199191+0000","last_clean":"2024-04-28T12:49:57.199191+0000","last_became_active":"2024-04-28T12:48:54.962294+0000","last_became_peered":"2024-04-28T12:48:54.962294+0000","last_unstale":"2024-04-28T12:49:57.199191+0000","last_undegraded":"2024-04-28T12:49:57.199191+0000","last_fullsized":"2024-04-28T12:49:57.199191+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:10:03.438962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199269+0000","last_change":"2024-04-28T12:48:54.967990+0000","last_active":"2024-04-28T12:49:57.199269+0000","last_peered":"2024-04-28T12:49:57.199269+0000","last_clean":"2024-04-28T12:49:57.199269+0000","last_became_active":"2024-04-28T12:48:54.967608+0000","last_became_peered":"2024-04-28T12:48:54.967608+0000","last_unstale":"2024-04-28T12:49:57.199269+0000","last_undegraded":"2024-04-28T12:49:57.199269+0000","last_fullsized":"2024-04-28T12:49:57.199269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:24:17.209965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198739+0000","last_change":"2024-04-28T12:48:54.974436+0000","last_active":"2024-04-28T12:49:57.198739+0000","last_peered":"2024-04-28T12:49:57.198739+0000","last_clean":"2024-04-28T12:49:57.198739+0000","last_became_active":"2024-04-28T12:48:54.973950+0000","last_became_peered":"2024-04-28T12:48:54.973950+0000","last_unstale":"2024-04-28T12:49:57.198739+0000","last_undegraded":"2024-04-28T12:49:57.198739+0000","last_fullsized":"2024-04-28T12:49:57.198739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:51:08.957853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195685+0000","last_change":"2024-04-28T12:48:54.979452+0000","last_active":"2024-04-28T12:49:57.195685+0000","last_peered":"2024-04-28T12:49:57.195685+0000","last_clean":"2024-04-28T12:49:57.195685+0000","last_became_active":"2024-04-28T12:48:54.979033+0000","last_became_peered":"2024-04-28T12:48:54.979033+0000","last_unstale":"2024-04-28T12:49:57.195685+0000","last_undegraded":"2024-04-28T12:49:57.195685+0000","last_fullsized":"2024-04-28T12:49:57.195685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:26:46.644763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.438059+0000","last_change":"2024-04-28T12:48:54.970775+0000","last_active":"2024-04-28T12:49:57.438059+0000","last_peered":"2024-04-28T12:49:57.438059+0000","last_clean":"2024-04-28T12:49:57.438059+0000","last_became_active":"2024-04-28T12:48:54.970414+0000","last_became_peered":"2024-04-28T12:48:54.970414+0000","last_unstale":"2024-04-28T12:49:57.438059+0000","last_undegraded":"2024-04-28T12:49:57.438059+0000","last_fullsized":"2024-04-28T12:49:57.438059+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:49:59.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193732+0000","last_change":"2024-04-28T12:48:54.983460+0000","last_active":"2024-04-28T12:49:57.193732+0000","last_peered":"2024-04-28T12:49:57.193732+0000","last_clean":"2024-04-28T12:49:57.193732+0000","last_became_active":"2024-04-28T12:48:54.983083+0000","last_became_peered":"2024-04-28T12:48:54.983083+0000","last_unstale":"2024-04-28T12:49:57.193732+0000","last_undegraded":"2024-04-28T12:49:57.193732+0000","last_fullsized":"2024-04-28T12:49:57.193732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:56:14.922227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193769+0000","last_change":"2024-04-28T12:48:59.198781+0000","last_active":"2024-04-28T12:49:57.193769+0000","last_peered":"2024-04-28T12:49:57.193769+0000","last_clean":"2024-04-28T12:49:57.193769+0000","last_became_active":"2024-04-28T12:48:59.198393+0000","last_became_peered":"2024-04-28T12:48:59.198393+0000","last_unstale":"2024-04-28T12:49:57.193769+0000","last_undegraded":"2024-04-28T12:49:57.193769+0000","last_fullsized":"2024-04-28T12:49:57.193769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:37:01.888857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:47.206103+0000","last_change":"2024-04-28T12:48:54.131557+0000","last_active":"2024-04-28T12:50:47.206103+0000","last_peered":"2024-04-28T12:50:47.206103+0000","last_clean":"2024-04-28T12:50:47.206103+0000","last_became_active":"2024-04-28T12:48:54.128561+0000","last_became_peered":"2024-04-28T12:48:54.128561+0000","last_unstale":"2024-04-28T12:50:47.206103+0000","last_undegraded":"2024-04-28T12:50:47.206103+0000","last_fullsized":"2024-04-28T12:50:47.206103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:19:41.169510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.212442+0000","last_change":"2024-04-28T12:48:54.978972+0000","last_active":"2024-04-28T12:49:57.212442+0000","last_peered":"2024-04-28T12:49:57.212442+0000","last_clean":"2024-04-28T12:49:57.212442+0000","last_became_active":"2024-04-28T12:48:54.977025+0000","last_became_peered":"2024-04-28T12:48:54.977025+0000","last_unstale":"2024-04-28T12:49:57.212442+0000","last_undegraded":"2024-04-28T12:49:57.212442+0000","last_fullsized":"2024-04-28T12:49:57.212442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:22:54.411569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193436+0000","last_change":"2024-04-28T12:48:59.193410+0000","last_active":"2024-04-28T12:49:57.193436+0000","last_peered":"2024-04-28T12:49:57.193436+0000","last_clean":"2024-04-28T12:49:57.193436+0000","last_became_active":"2024-04-28T12:48:59.192864+0000","last_became_peered":"2024-04-28T12:48:59.192864+0000","last_unstale":"2024-04-28T12:49:57.193436+0000","last_undegraded":"2024-04-28T12:49:57.193436+0000","last_fullsized":"2024-04-28T12:49:57.193436+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:21:18.783650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.212024+0000","last_change":"2024-04-28T12:48:54.132973+0000","last_active":"2024-04-28T12:49:57.212024+0000","last_peered":"2024-04-28T12:49:57.212024+0000","last_clean":"2024-04-28T12:49:57.212024+0000","last_became_active":"2024-04-28T12:48:54.132590+0000","last_became_peered":"2024-04-28T12:48:54.132590+0000","last_unstale":"2024-04-28T12:49:57.212024+0000","last_undegraded":"2024-04-28T12:49:57.212024+0000","last_fullsized":"2024-04-28T12:49:57.212024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:50:52.634742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.203743+0000","last_change":"2024-04-28T12:48:54.975902+0000","last_active":"2024-04-28T12:49:57.203743+0000","last_peered":"2024-04-28T12:49:57.203743+0000","last_clean":"2024-04-28T12:49:57.203743+0000","last_became_active":"2024-04-28T12:48:54.975488+0000","last_became_peered":"2024-04-28T12:48:54.975488+0000","last_unstale":"2024-04-28T12:49:57.203743+0000","last_undegraded":"2024-04-28T12:49:57.203743+0000","last_fullsized":"2024-04-28T12:49:57.203743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:15:10.348412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193619+0000","last_change":"2024-04-28T12:48:59.187912+0000","last_active":"2024-04-28T12:49:57.193619+0000","last_peered":"2024-04-28T12:49:57.193619+0000","last_clean":"2024-04-28T12:49:57.193619+0000","last_became_active":"2024-04-28T12:48:59.187427+0000","last_became_peered":"2024-04-28T12:48:59.187427+0000","last_unstale":"2024-04-28T12:49:57.193619+0000","last_undegraded":"2024-04-28T12:49:57.193619+0000","last_fullsized":"2024-04-28T12:49:57.193619+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:27:53.137136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198094+0000","last_change":"2024-04-28T12:48:54.117132+0000","last_active":"2024-04-28T12:49:57.198094+0000","last_peered":"2024-04-28T12:49:57.198094+0000","last_clean":"2024-04-28T12:49:57.198094+0000","last_became_active":"2024-04-28T12:48:54.116805+0000","last_became_peered":"2024-04-28T12:48:54.116805+0000","last_unstale":"2024-04-28T12:49:57.198094+0000","last_undegraded":"2024-04-28T12:49:57.198094+0000","last_fullsized":"2024-04-28T12:49:57.198094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:22:52.015199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199040+0000","last_change":"2024-04-28T12:48:54.972975+0000","last_active":"2024-04-28T12:49:57.199040+0000","last_peered":"2024-04-28T12:49:57.199040+0000","last_clean":"2024-04-28T12:49:57.199040+0000","last_became_active":"2024-04-28T12:48:54.972679+0000","last_became_peered":"2024-04-28T12:48:54.972679+0000","last_unstale":"2024-04-28T12:49:57.199040+0000","last_undegraded":"2024-04-28T12:49:57.199040+0000","last_fullsized":"2024-04-28T12:49:57.199040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:50:22.032459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193501+0000","last_change":"2024-04-28T12:48:59.187828+0000","last_active":"2024-04-28T12:49:57.193501+0000","last_peered":"2024-04-28T12:49:57.193501+0000","last_clean":"2024-04-28T12:49:57.193501+0000","last_became_active":"2024-04-28T12:48:59.187435+0000","last_became_peered":"2024-04-28T12:48:59.187435+0000","last_unstale":"2024-04-28T12:49:57.193501+0000","last_undegraded":"2024-04-28T12:49:57.193501+0000","last_fullsized":"2024-04-28T12:49:57.193501+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:45:22.909378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197990+0000","last_change":"2024-04-28T12:48:54.112455+0000","last_active":"2024-04-28T12:49:57.197990+0000","last_peered":"2024-04-28T12:49:57.197990+0000","last_clean":"2024-04-28T12:49:57.197990+0000","last_became_active":"2024-04-28T12:48:54.111927+0000","last_became_peered":"2024-04-28T12:48:54.111927+0000","last_unstale":"2024-04-28T12:49:57.197990+0000","last_undegraded":"2024-04-28T12:49:57.197990+0000","last_fullsized":"2024-04-28T12:49:57.197990+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:32:54.650426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195829+0000","last_change":"2024-04-28T12:48:54.959687+0000","last_active":"2024-04-28T12:49:57.195829+0000","last_peered":"2024-04-28T12:49:57.195829+0000","last_clean":"2024-04-28T12:49:57.195829+0000","last_became_active":"2024-04-28T12:48:54.959355+0000","last_became_peered":"2024-04-28T12:48:54.959355+0000","last_unstale":"2024-04-28T12:49:57.195829+0000","last_undegraded":"2024-04-28T12:49:57.195829+0000","last_fullsized":"2024-04-28T12:49:57.195829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:53:53.919155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193534+0000","last_change":"2024-04-28T12:48:59.174209+0000","last_active":"2024-04-28T12:49:57.193534+0000","last_peered":"2024-04-28T12:49:57.193534+0000","last_clean":"2024-04-28T12:49:57.193534+0000","last_became_active":"2024-04-28T12:48:59.173700+0000","last_became_peered":"2024-04-28T12:48:59.173700+0000","last_unstale":"2024-04-28T12:49:57.193534+0000","last_undegraded":"2024-04-28T12:49:57.193534+0000","last_fullsized":"2024-04-28T12:49:57.193534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:19:11.820482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:47.141864+0000","last_change":"2024-04-28T12:48:54.118061+0000","last_active":"2024-04-28T12:50:47.141864+0000","last_peered":"2024-04-28T12:50:47.141864+0000","last_clean":"2024-04-28T12:50:47.141864+0000","last_became_active":"2024-04-28T12:48:54.117663+0000","last_became_peered":"2024-04-28T12:48:54.117663+0000","last_unstale":"2024-04-28T12:50:47.141864+0000","last_undegraded":"2024-04-28T12:50:47.141864+0000","last_fullsized":"2024-04-28T12:50:47.141864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:41:45.206989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197749+0000","last_change":"2024-04-28T12:48:54.975039+0000","last_active":"2024-04-28T12:49:57.197749+0000","last_peered":"2024-04-28T12:49:57.197749+0000","last_clean":"2024-04-28T12:49:57.197749+0000","last_became_active":"2024-04-28T12:48:54.974589+0000","last_became_peered":"2024-04-28T12:48:54.974589+0000","last_unstale":"2024-04-28T12:49:57.197749+0000","last_undegraded":"2024-04-28T12:49:57.197749+0000","last_fullsized":"2024-04-28T12:49:57.197749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:24:30.550231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193680+0000","last_change":"2024-04-28T12:48:59.183009+0000","last_active":"2024-04-28T12:49:57.193680+0000","last_peered":"2024-04-28T12:49:57.193680+0000","last_clean":"2024-04-28T12:49:57.193680+0000","last_became_active":"2024-04-28T12:48:59.182641+0000","last_became_peered":"2024-04-28T12:48:59.182641+0000","last_unstale":"2024-04-28T12:49:57.193680+0000","last_undegraded":"2024-04-28T12:49:57.193680+0000","last_fullsized":"2024-04-28T12:49:57.193680+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:21.059396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436335+0000","last_change":"2024-04-28T12:48:54.120770+0000","last_active":"2024-04-28T12:49:57.436335+0000","last_peered":"2024-04-28T12:49:57.436335+0000","last_clean":"2024-04-28T12:49:57.436335+0000","last_became_active":"2024-04-28T12:48:54.120429+0000","last_became_peered":"2024-04-28T12:48:54.120429+0000","last_unstale":"2024-04-28T12:49:57.436335+0000","last_undegraded":"2024-04-28T12:49:57.436335+0000","last_fullsized":"2024-04-28T12:49:57.436335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:27:28.172879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195634+0000","last_change":"2024-04-28T12:48:54.982623+0000","last_active":"2024-04-28T12:49:57.195634+0000","last_peered":"2024-04-28T12:49:57.195634+0000","last_clean":"2024-04-28T12:49:57.195634+0000","last_became_active":"2024-04-28T12:48:54.982225+0000","last_became_peered":"2024-04-28T12:48:54.982225+0000","last_unstale":"2024-04-28T12:49:57.195634+0000","last_undegraded":"2024-04-28T12:49:57.195634+0000","last_fullsized":"2024-04-28T12:49:57.195634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:40:17.652783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195291+0000","last_change":"2024-04-28T12:48:59.188489+0000","last_active":"2024-04-28T12:49:57.195291+0000","last_peered":"2024-04-28T12:49:57.195291+0000","last_clean":"2024-04-28T12:49:57.195291+0000","last_became_active":"2024-04-28T12:48:59.188008+0000","last_became_peered":"2024-04-28T12:48:59.188008+0000","last_unstale":"2024-04-28T12:49:57.195291+0000","last_undegraded":"2024-04-28T12:49:57.195291+0000","last_fullsized":"2024-04-28T12:49:57.195291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:34:55.971411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.206549+0000","last_change":"2024-04-28T12:48:54.137567+0000","last_active":"2024-04-28T12:49:57.206549+0000","last_peered":"2024-04-28T12:49:57.206549+0000","last_clean":"2024-04-28T12:49:57.206549+0000","last_became_active":"2024-04-28T12:48:54.137009+0000","last_became_peered":"2024-04-28T12:48:54.137009+0000","last_unstale":"2024-04-28T12:49:57.206549+0000","last_undegraded":"2024-04-28T12:49:57.206549+0000","last_fullsized":"2024-04-28T12:49:57.206549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:01:30.909066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.211593+0000","last_change":"2024-04-28T12:48:54.974583+0000","last_active":"2024-04-28T12:49:57.211593+0000","last_peered":"2024-04-28T12:49:57.211593+0000","last_clean":"2024-04-28T12:49:57.211593+0000","last_became_active":"2024-04-28T12:48:54.974217+0000","last_became_peered":"2024-04-28T12:48:54.974217+0000","last_unstale":"2024-04-28T12:49:57.211593+0000","last_undegraded":"2024-04-28T12:49:57.211593+0000","last_fullsized":"2024-04-28T12:49:57.211593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:52:05.768593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.201305+0000","last_change":"2024-04-28T12:48:59.185263+0000","last_active":"2024-04-28T12:49:57.201305+0000","last_peered":"2024-04-28T12:49:57.201305+0000","last_clean":"2024-04-28T12:49:57.201305+0000","last_became_active":"2024-04-28T12:48:59.184423+0000","last_became_peered":"2024-04-28T12:48:59.184423+0000","last_unstale":"2024-04-28T12:49:57.201305+0000","last_undegraded":"2024-04-28T12:49:57.201305+0000","last_fullsized":"2024-04-28T12:49:57.201305+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:31:01.029985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195623+0000","last_change":"2024-04-28T12:48:54.119423+0000","last_active":"2024-04-28T12:49:57.195623+0000","last_peered":"2024-04-28T12:49:57.195623+0000","last_clean":"2024-04-28T12:49:57.195623+0000","last_became_active":"2024-04-28T12:48:54.119110+0000","last_became_peered":"2024-04-28T12:48:54.119110+0000","last_unstale":"2024-04-28T12:49:57.195623+0000","last_undegraded":"2024-04-28T12:49:57.195623+0000","last_fullsized":"2024-04-28T12:49:57.195623+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:16:33.181397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437777+0000","last_change":"2024-04-28T12:48:54.983978+0000","last_active":"2024-04-28T12:49:57.437777+0000","last_peered":"2024-04-28T12:49:57.437777+0000","last_clean":"2024-04-28T12:49:57.437777+0000","last_became_active":"2024-04-28T12:48:54.982386+0000","last_became_peered":"2024-04-28T12:48:54.982386+0000","last_unstale":"2024-04-28T12:49:57.437777+0000","last_undegraded":"2024-04-28T12:49:57.437777+0000","last_fullsized":"2024-04-28T12:49:57.437777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:51:23.849403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.208765+0000","last_change":"2024-04-28T12:48:59.187771+0000","last_active":"2024-04-28T12:49:57.208765+0000","last_peered":"2024-04-28T12:49:57.208765+0000","last_clean":"2024-04-28T12:49:57.208765+0000","last_became_active":"2024-04-28T12:48:59.187265+0000","last_became_peered":"2024-04-28T12:48:59.187265+0000","last_unstale":"2024-04-28T12:49:57.208765+0000","last_undegraded":"2024-04-28T12:49:57.208765+0000","last_fullsized":"2024-04-28T12:49:57.208765+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:51.211998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":194,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:50.211004+0000","last_change":"2024-04-28T12:48:54.135666+0000","last_active":"2024-04-28T12:50:50.211004+0000","last_peered":"2024-04-28T12:50:50.211004+0000","last_clean":"2024-04-28T12:50:50.211004+0000","last_became_active":"2024-04-28T12:48:54.135373+0000","last_became_peered":"2024-04-28T12:48:54.135373+0000","last_unstale":"2024-04-28T12:50:50.211004+0000","last_undegraded":"2024-04-28T12:50:50.211004+0000","last_fullsized":"2024-04-28T12:50:50.211004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:41:52.455552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":163,"num_read_kb":163,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199231+0000","last_change":"2024-04-28T12:48:54.976474+0000","last_active":"2024-04-28T12:49:57.199231+0000","last_peered":"2024-04-28T12:49:57.199231+0000","last_clean":"2024-04-28T12:49:57.199231+0000","last_became_active":"2024-04-28T12:48:54.976133+0000","last_became_peered":"2024-04-28T12:48:54.976133+0000","last_unstale":"2024-04-28T12:49:57.199231+0000","last_undegraded":"2024-04-28T12:49:57.199231+0000","last_fullsized":"2024-04-28T12:49:57.199231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:24:19.046535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437452+0000","last_change":"2024-04-28T12:48:59.181836+0000","last_active":"2024-04-28T12:49:57.437452+0000","last_peered":"2024-04-28T12:49:57.437452+0000","last_clean":"2024-04-28T12:49:57.437452+0000","last_became_active":"2024-04-28T12:48:59.177499+0000","last_became_peered":"2024-04-28T12:48:59.177499+0000","last_unstale":"2024-04-28T12:49:57.437452+0000","last_undegraded":"2024-04-28T12:49:57.437452+0000","last_fullsized":"2024-04-28T12:49:57.437452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:36:37.981250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198033+0000","last_change":"2024-04-28T12:48:54.118519+0000","last_active":"2024-04-28T12:49:57.198033+0000","last_peered":"2024-04-28T12:49:57.198033+0000","last_clean":"2024-04-28T12:49:57.198033+0000","last_became_active":"2024-04-28T12:48:54.116755+0000","last_became_peered":"2024-04-28T12:48:54.116755+0000","last_unstale":"2024-04-28T12:49:57.198033+0000","last_undegraded":"2024-04-28T12:49:57.198033+0000","last_fullsized":"2024-04-28T12:49:57.198033+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:05:46.261098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.210551+0000","last_change":"2024-04-28T12:49:57.210551+0000","last_active":"2024-04-28T12:49:57.210551+0000","last_peered":"2024-04-28T12:49:57.210551+0000","last_clean":"2024-04-28T12:49:57.210551+0000","last_became_active":"2024-04-28T12:49:57.209805+0000","last_became_peered":"2024-04-28T12:49:57.209805+0000","last_unstale":"2024-04-28T12:49:57.210551+0000","last_undegraded":"2024-04-28T12:49:57.210551+0000","last_fullsized":"2024-04-28T12:49:57.210551+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:30:29.206125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193870+0000","last_change":"2024-04-28T12:48:59.179857+0000","last_active":"2024-04-28T12:49:57.193870+0000","last_peered":"2024-04-28T12:49:57.193870+0000","last_clean":"2024-04-28T12:49:57.193870+0000","last_became_active":"2024-04-28T12:48:59.179412+0000","last_became_peered":"2024-04-28T12:48:59.179412+0000","last_unstale":"2024-04-28T12:49:57.193870+0000","last_undegraded":"2024-04-28T12:49:57.193870+0000","last_fullsized":"2024-04-28T12:49:57.193870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:28:08.883832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195630+0000","last_change":"2024-04-28T12:48:54.123771+0000","last_active":"2024-04-28T12:49:57.195630+0000","last_peered":"2024-04-28T12:49:57.195630+0000","last_clean":"2024-04-28T12:49:57.195630+0000","last_became_active":"2024-04-28T12:48:54.123524+0000","last_became_peered":"2024-04-28T12:48:54.123524+0000","last_unstale":"2024-04-28T12:49:57.195630+0000","last_undegraded":"2024-04-28T12:49:57.195630+0000","last_fullsized":"2024-04-28T12:49:57.195630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:59:59.982340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199003+0000","last_change":"2024-04-28T12:48:54.966190+0000","last_active":"2024-04-28T12:49:57.199003+0000","last_peered":"2024-04-28T12:49:57.199003+0000","last_clean":"2024-04-28T12:49:57.199003+0000","last_became_active":"2024-04-28T12:48:54.965854+0000","last_became_peered":"2024-04-28T12:48:54.965854+0000","last_unstale":"2024-04-28T12:49:57.199003+0000","last_undegraded":"2024-04-28T12:49:57.199003+0000","last_fullsized":"2024-04-28T12:49:57.199003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:34:16.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437785+0000","last_change":"2024-04-28T12:48:59.182747+0000","last_active":"2024-04-28T12:49:57.437785+0000","last_peered":"2024-04-28T12:49:57.437785+0000","last_clean":"2024-04-28T12:49:57.437785+0000","last_became_active":"2024-04-28T12:48:59.179610+0000","last_became_peered":"2024-04-28T12:48:59.179610+0000","last_unstale":"2024-04-28T12:49:57.437785+0000","last_undegraded":"2024-04-28T12:49:57.437785+0000","last_fullsized":"2024-04-28T12:49:57.437785+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:19:19.603188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'32","reported_seq":229,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:51.146098+0000","last_change":"2024-04-28T12:48:54.116386+0000","last_active":"2024-04-28T12:50:51.146098+0000","last_peered":"2024-04-28T12:50:51.146098+0000","last_clean":"2024-04-28T12:50:51.146098+0000","last_became_active":"2024-04-28T12:48:54.115547+0000","last_became_peered":"2024-04-28T12:48:54.115547+0000","last_unstale":"2024-04-28T12:50:51.146098+0000","last_undegraded":"2024-04-28T12:50:51.146098+0000","last_fullsized":"2024-04-28T12:50:51.146098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:10:17.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51771,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":170,"num_read_kb":135,"num_write":34,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197749+0000","last_change":"2024-04-28T12:48:54.962221+0000","last_active":"2024-04-28T12:49:57.197749+0000","last_peered":"2024-04-28T12:49:57.197749+0000","last_clean":"2024-04-28T12:49:57.197749+0000","last_became_active":"2024-04-28T12:48:54.961876+0000","last_became_peered":"2024-04-28T12:48:54.961876+0000","last_unstale":"2024-04-28T12:49:57.197749+0000","last_undegraded":"2024-04-28T12:49:57.197749+0000","last_fullsized":"2024-04-28T12:49:57.197749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:22:38.067688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.196171+0000","last_change":"2024-04-28T12:48:59.193852+0000","last_active":"2024-04-28T12:49:57.196171+0000","last_peered":"2024-04-28T12:49:57.196171+0000","last_clean":"2024-04-28T12:49:57.196171+0000","last_became_active":"2024-04-28T12:48:59.193100+0000","last_became_peered":"2024-04-28T12:48:59.193100+0000","last_unstale":"2024-04-28T12:49:57.196171+0000","last_undegraded":"2024-04-28T12:49:57.196171+0000","last_fullsized":"2024-04-28T12:49:57.196171+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:35:26.732962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193667+0000","last_change":"2024-04-28T12:48:54.129317+0000","last_active":"2024-04-28T12:49:57.193667+0000","last_peered":"2024-04-28T12:49:57.193667+0000","last_clean":"2024-04-28T12:49:57.193667+0000","last_became_active":"2024-04-28T12:48:54.128980+0000","last_became_peered":"2024-04-28T12:48:54.128980+0000","last_unstale":"2024-04-28T12:49:57.193667+0000","last_undegraded":"2024-04-28T12:49:57.193667+0000","last_fullsized":"2024-04-28T12:49:57.193667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:02:34.543478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202921+0000","last_change":"2024-04-28T12:48:54.983576+0000","last_active":"2024-04-28T12:49:57.202921+0000","last_peered":"2024-04-28T12:49:57.202921+0000","last_clean":"2024-04-28T12:49:57.202921+0000","last_became_active":"2024-04-28T12:48:54.983213+0000","last_became_peered":"2024-04-28T12:48:54.983213+0000","last_unstale":"2024-04-28T12:49:57.202921+0000","last_undegraded":"2024-04-28T12:49:57.202921+0000","last_fullsized":"2024-04-28T12:49:57.202921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:22:10.901393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197230+0000","last_change":"2024-04-28T12:48:59.191085+0000","last_active":"2024-04-28T12:49:57.197230+0000","last_peered":"2024-04-28T12:49:57.197230+0000","last_clean":"2024-04-28T12:49:57.197230+0000","last_became_active":"2024-04-28T12:48:59.190431+0000","last_became_peered":"2024-04-28T12:48:59.190431+0000","last_unstale":"2024-04-28T12:49:57.197230+0000","last_undegraded":"2024-04-28T12:49:57.197230+0000","last_fullsized":"2024-04-28T12:49:57.197230+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:13:30.990816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.196492+0000","last_change":"2024-04-28T12:48:54.129634+0000","last_active":"2024-04-28T12:49:57.196492+0000","last_peered":"2024-04-28T12:49:57.196492+0000","last_clean":"2024-04-28T12:49:57.196492+0000","last_became_active":"2024-04-28T12:48:54.129250+0000","last_became_peered":"2024-04-28T12:48:54.129250+0000","last_unstale":"2024-04-28T12:49:57.196492+0000","last_undegraded":"2024-04-28T12:49:57.196492+0000","last_fullsized":"2024-04-28T12:49:57.196492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:48:23.828967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197574+0000","last_change":"2024-04-28T12:48:54.978551+0000","last_active":"2024-04-28T12:49:57.197574+0000","last_peered":"2024-04-28T12:49:57.197574+0000","last_clean":"2024-04-28T12:49:57.197574+0000","last_became_active":"2024-04-28T12:48:54.978193+0000","last_became_peered":"2024-04-28T12:48:54.978193+0000","last_unstale":"2024-04-28T12:49:57.197574+0000","last_undegraded":"2024-04-28T12:49:57.197574+0000","last_fullsized":"2024-04-28T12:49:57.197574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:05:31.289174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437926+0000","last_change":"2024-04-28T12:48:59.195185+0000","last_active":"2024-04-28T12:49:57.437926+0000","last_peered":"2024-04-28T12:49:57.437926+0000","last_clean":"2024-04-28T12:49:57.437926+0000","last_became_active":"2024-04-28T12:48:59.194694+0000","last_became_peered":"2024-04-28T12:48:59.194694+0000","last_unstale":"2024-04-28T12:49:57.437926+0000","last_undegraded":"2024-04-28T12:49:57.437926+0000","last_fullsized":"2024-04-28T12:49:57.437926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:20:32.584910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:47.178087+0000","last_change":"2024-04-28T12:48:54.115249+0000","last_active":"2024-04-28T12:50:47.178087+0000","last_peered":"2024-04-28T12:50:47.178087+0000","last_clean":"2024-04-28T12:50:47.178087+0000","last_became_active":"2024-04-28T12:48:54.114888+0000","last_became_peered":"2024-04-28T12:48:54.114888+0000","last_unstale":"2024-04-28T12:50:47.178087+0000","last_undegraded":"2024-04-28T12:50:47.178087+0000","last_fullsized":"2024-04-28T12:50:47.178087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:47:30.899021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197425+0000","last_change":"2024-04-28T12:48:54.960038+0000","last_active":"2024-04-28T12:49:57.197425+0000","last_peered":"2024-04-28T12:49:57.197425+0000","last_clean":"2024-04-28T12:49:57.197425+0000","last_became_active":"2024-04-28T12:48:54.959710+0000","last_became_peered":"2024-04-28T12:48:54.959710+0000","last_unstale":"2024-04-28T12:49:57.197425+0000","last_undegraded":"2024-04-28T12:49:57.197425+0000","last_fullsized":"2024-04-28T12:49:57.197425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:41:13.538731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.203299+0000","last_change":"2024-04-28T12:48:59.197523+0000","last_active":"2024-04-28T12:49:57.203299+0000","last_peered":"2024-04-28T12:49:57.203299+0000","last_clean":"2024-04-28T12:49:57.203299+0000","last_became_active":"2024-04-28T12:48:59.197088+0000","last_became_peered":"2024-04-28T12:48:59.197088+0000","last_unstale":"2024-04-28T12:49:57.203299+0000","last_undegraded":"2024-04-28T12:49:57.203299+0000","last_fullsized":"2024-04-28T12:49:57.203299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:53:33.391084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197875+0000","last_change":"2024-04-28T12:48:54.130359+0000","last_active":"2024-04-28T12:49:57.197875+0000","last_peered":"2024-04-28T12:49:57.197875+0000","last_clean":"2024-04-28T12:49:57.197875+0000","last_became_active":"2024-04-28T12:48:54.130043+0000","last_became_peered":"2024-04-28T12:48:54.130043+0000","last_unstale":"2024-04-28T12:49:57.197875+0000","last_undegraded":"2024-04-28T12:49:57.197875+0000","last_fullsized":"2024-04-28T12:49:57.197875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:05:41.353359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.200138+0000","last_change":"2024-04-28T12:49:57.200138+0000","last_active":"2024-04-28T12:49:57.200138+0000","last_peered":"2024-04-28T12:49:57.200138+0000","last_clean":"2024-04-28T12:49:57.200138+0000","last_became_active":"2024-04-28T12:49:57.199869+0000","last_became_peered":"2024-04-28T12:49:57.199869+0000","last_unstale":"2024-04-28T12:49:57.200138+0000","last_undegraded":"2024-04-28T12:49:57.200138+0000","last_fullsized":"2024-04-28T12:49:57.200138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:52:01.476631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.196809+0000","last_change":"2024-04-28T12:48:59.193963+0000","last_active":"2024-04-28T12:49:57.196809+0000","last_peered":"2024-04-28T12:49:57.196809+0000","last_clean":"2024-04-28T12:49:57.196809+0000","last_became_active":"2024-04-28T12:48:59.193551+0000","last_became_peered":"2024-04-28T12:48:59.193551+0000","last_unstale":"2024-04-28T12:49:57.196809+0000","last_undegraded":"2024-04-28T12:49:57.196809+0000","last_fullsized":"2024-04-28T12:49:57.196809+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:24:29.474296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198030+0000","last_change":"2024-04-28T12:48:54.109277+0000","last_active":"2024-04-28T12:49:57.198030+0000","last_peered":"2024-04-28T12:49:57.198030+0000","last_clean":"2024-04-28T12:49:57.198030+0000","last_became_active":"2024-04-28T12:48:54.108931+0000","last_became_peered":"2024-04-28T12:48:54.108931+0000","last_unstale":"2024-04-28T12:49:57.198030+0000","last_undegraded":"2024-04-28T12:49:57.198030+0000","last_fullsized":"2024-04-28T12:49:57.198030+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:56:29.683311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195331+0000","last_change":"2024-04-28T12:48:54.961014+0000","last_active":"2024-04-28T12:49:57.195331+0000","last_peered":"2024-04-28T12:49:57.195331+0000","last_clean":"2024-04-28T12:49:57.195331+0000","last_became_active":"2024-04-28T12:48:54.960648+0000","last_became_peered":"2024-04-28T12:48:54.960648+0000","last_unstale":"2024-04-28T12:49:57.195331+0000","last_undegraded":"2024-04-28T12:49:57.195331+0000","last_fullsized":"2024-04-28T12:49:57.195331+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:02:42.794451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197715+0000","last_change":"2024-04-28T12:48:59.185458+0000","last_active":"2024-04-28T12:49:57.197715+0000","last_peered":"2024-04-28T12:49:57.197715+0000","last_clean":"2024-04-28T12:49:57.197715+0000","last_became_active":"2024-04-28T12:48:59.184959+0000","last_became_peered":"2024-04-28T12:48:59.184959+0000","last_unstale":"2024-04-28T12:49:57.197715+0000","last_undegraded":"2024-04-28T12:49:57.197715+0000","last_fullsized":"2024-04-28T12:49:57.197715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:46:45.723374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.438375+0000","last_change":"2024-04-28T12:48:54.134303+0000","last_active":"2024-04-28T12:49:57.438375+0000","last_peered":"2024-04-28T12:49:57.438375+0000","last_clean":"2024-04-28T12:49:57.438375+0000","last_became_active":"2024-04-28T12:48:54.133728+0000","last_became_peered":"2024-04-28T12:48:54.133728+0000","last_unstale":"2024-04-28T12:49:57.438375+0000","last_undegraded":"2024-04-28T12:49:57.438375+0000","last_fullsized":"2024-04-28T12:49:57.438375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:44:16.444168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436554+0000","last_change":"2024-04-28T12:48:54.983312+0000","last_active":"2024-04-28T12:49:57.436554+0000","last_peered":"2024-04-28T12:49:57.436554+0000","last_clean":"2024-04-28T12:49:57.436554+0000","last_became_active":"2024-04-28T12:48:54.982928+0000","last_became_peered":"2024-04-28T12:48:54.982928+0000","last_unstale":"2024-04-28T12:49:57.436554+0000","last_undegraded":"2024-04-28T12:49:57.436554+0000","last_fullsized":"2024-04-28T12:49:57.436554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:18:22.411048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202012+0000","last_change":"2024-04-28T12:48:59.187641+0000","last_active":"2024-04-28T12:49:57.202012+0000","last_peered":"2024-04-28T12:49:57.202012+0000","last_clean":"2024-04-28T12:49:57.202012+0000","last_became_active":"2024-04-28T12:48:59.186003+0000","last_became_peered":"2024-04-28T12:48:59.186003+0000","last_unstale":"2024-04-28T12:49:57.202012+0000","last_undegraded":"2024-04-28T12:49:57.202012+0000","last_fullsized":"2024-04-28T12:49:57.202012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:50:34.968976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":113,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:48.209390+0000","last_change":"2024-04-28T12:48:54.122909+0000","last_active":"2024-04-28T12:50:48.209390+0000","last_peered":"2024-04-28T12:50:48.209390+0000","last_clean":"2024-04-28T12:50:48.209390+0000","last_became_active":"2024-04-28T12:48:54.122661+0000","last_became_peered":"2024-04-28T12:48:54.122661+0000","last_unstale":"2024-04-28T12:50:48.209390+0000","last_undegraded":"2024-04-28T12:50:48.209390+0000","last_fullsized":"2024-04-28T12:50:48.209390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:43:06.119038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.436368+0000","last_change":"2024-04-28T12:48:54.956618+0000","last_active":"2024-04-28T12:49:57.436368+0000","last_peered":"2024-04-28T12:49:57.436368+0000","last_clean":"2024-04-28T12:49:57.436368+0000","last_became_active":"2024-04-28T12:48:54.956227+0000","last_became_peered":"2024-04-28T12:48:54.956227+0000","last_unstale":"2024-04-28T12:49:57.436368+0000","last_undegraded":"2024-04-28T12:49:57.436368+0000","last_fullsized":"2024-04-28T12:49:57.436368+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:38:02.820771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.203682+0000","last_change":"2024-04-28T12:48:59.187017+0000","last_active":"2024-04-28T12:49:57.203682+0000","last_peered":"2024-04-28T12:49:57.203682+0000","last_clean":"2024-04-28T12:49:57.203682+0000","last_became_active":"2024-04-28T12:48:59.186623+0000","last_became_peered":"2024-04-28T12:48:59.186623+0000","last_unstale":"2024-04-28T12:49:57.203682+0000","last_undegraded":"2024-04-28T12:49:57.203682+0000","last_fullsized":"2024-04-28T12:49:57.203682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:26:53.631400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195818+0000","last_change":"2024-04-28T12:48:54.957348+0000","last_active":"2024-04-28T12:49:57.195818+0000","last_peered":"2024-04-28T12:49:57.195818+0000","last_clean":"2024-04-28T12:49:57.195818+0000","last_became_active":"2024-04-28T12:48:54.956890+0000","last_became_peered":"2024-04-28T12:48:54.956890+0000","last_unstale":"2024-04-28T12:49:57.195818+0000","last_undegraded":"2024-04-28T12:49:57.195818+0000","last_fullsized":"2024-04-28T12:49:57.195818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:41:24.550175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:46.175639+0000","last_change":"2024-04-28T12:48:54.133596+0000","last_active":"2024-04-28T12:50:46.175639+0000","last_peered":"2024-04-28T12:50:46.175639+0000","last_clean":"2024-04-28T12:50:46.175639+0000","last_became_active":"2024-04-28T12:48:54.131632+0000","last_became_peered":"2024-04-28T12:48:54.131632+0000","last_unstale":"2024-04-28T12:50:46.175639+0000","last_undegraded":"2024-04-28T12:50:46.175639+0000","last_fullsized":"2024-04-28T12:50:46.175639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:42:39.352397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197920+0000","last_change":"2024-04-28T12:48:59.174117+0000","last_active":"2024-04-28T12:49:57.197920+0000","last_peered":"2024-04-28T12:49:57.197920+0000","last_clean":"2024-04-28T12:49:57.197920+0000","last_became_active":"2024-04-28T12:48:59.173699+0000","last_became_peered":"2024-04-28T12:48:59.173699+0000","last_unstale":"2024-04-28T12:49:57.197920+0000","last_undegraded":"2024-04-28T12:49:57.197920+0000","last_fullsized":"2024-04-28T12:49:57.197920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:41:51.186413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.211167+0000","last_change":"2024-04-28T12:48:54.978658+0000","last_active":"2024-04-28T12:49:57.211167+0000","last_peered":"2024-04-28T12:49:57.211167+0000","last_clean":"2024-04-28T12:49:57.211167+0000","last_became_active":"2024-04-28T12:48:54.978113+0000","last_became_peered":"2024-04-28T12:48:54.978113+0000","last_unstale":"2024-04-28T12:49:57.211167+0000","last_undegraded":"2024-04-28T12:49:57.211167+0000","last_fullsized":"2024-04-28T12:49:57.211167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:33:50.716489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193767+0000","last_change":"2024-04-28T12:48:54.130301+0000","last_active":"2024-04-28T12:49:57.193767+0000","last_peered":"2024-04-28T12:49:57.193767+0000","last_clean":"2024-04-28T12:49:57.193767+0000","last_became_active":"2024-04-28T12:48:54.130027+0000","last_became_peered":"2024-04-28T12:48:54.130027+0000","last_unstale":"2024-04-28T12:49:57.193767+0000","last_undegraded":"2024-04-28T12:49:57.193767+0000","last_fullsized":"2024-04-28T12:49:57.193767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:30:03.883731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":385,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197343+0000","last_change":"2024-04-28T12:47:34.946674+0000","last_active":"2024-04-28T12:49:57.197343+0000","last_peered":"2024-04-28T12:49:57.197343+0000","last_clean":"2024-04-28T12:49:57.197343+0000","last_became_active":"2024-04-28T12:47:34.946149+0000","last_became_peered":"2024-04-28T12:47:34.946149+0000","last_unstale":"2024-04-28T12:49:57.197343+0000","last_undegraded":"2024-04-28T12:49:57.197343+0000","last_fullsized":"2024-04-28T12:49:57.197343+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:43:40.694612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:43:40.694612+0000","last_clean_scrub_stamp":"2024-04-28T12:43:40.694612+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:25:44.776805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195704+0000","last_change":"2024-04-28T12:48:59.190783+0000","last_active":"2024-04-28T12:49:57.195704+0000","last_peered":"2024-04-28T12:49:57.195704+0000","last_clean":"2024-04-28T12:49:57.195704+0000","last_became_active":"2024-04-28T12:48:59.190383+0000","last_became_peered":"2024-04-28T12:48:59.190383+0000","last_unstale":"2024-04-28T12:49:57.195704+0000","last_undegraded":"2024-04-28T12:49:57.195704+0000","last_fullsized":"2024-04-28T12:49:57.195704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:39:41.711227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.439823+0000","last_change":"2024-04-28T12:49:57.439823+0000","last_active":"2024-04-28T12:49:57.439823+0000","last_peered":"2024-04-28T12:49:57.439823+0000","last_clean":"2024-04-28T12:49:57.439823+0000","last_became_active":"2024-04-28T12:49:57.439112+0000","last_became_peered":"2024-04-28T12:49:57.439112+0000","last_unstale":"2024-04-28T12:49:57.439823+0000","last_undegraded":"2024-04-28T12:49:57.439823+0000","last_fullsized":"2024-04-28T12:49:57.439823+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:36:28.295692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197627+0000","last_change":"2024-04-28T12:48:54.116384+0000","last_active":"2024-04-28T12:49:57.197627+0000","last_peered":"2024-04-28T12:49:57.197627+0000","last_clean":"2024-04-28T12:49:57.197627+0000","last_became_active":"2024-04-28T12:48:54.114362+0000","last_became_peered":"2024-04-28T12:48:54.114362+0000","last_unstale":"2024-04-28T12:49:57.197627+0000","last_undegraded":"2024-04-28T12:49:57.197627+0000","last_fullsized":"2024-04-28T12:49:57.197627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:17:20.406641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195864+0000","last_change":"2024-04-28T12:48:59.181370+0000","last_active":"2024-04-28T12:49:57.195864+0000","last_peered":"2024-04-28T12:49:57.195864+0000","last_clean":"2024-04-28T12:49:57.195864+0000","last_became_active":"2024-04-28T12:48:59.180852+0000","last_became_peered":"2024-04-28T12:48:59.180852+0000","last_unstale":"2024-04-28T12:49:57.195864+0000","last_undegraded":"2024-04-28T12:49:57.195864+0000","last_fullsized":"2024-04-28T12:49:57.195864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:08:55.532273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198025+0000","last_change":"2024-04-28T12:48:54.129792+0000","last_active":"2024-04-28T12:49:57.198025+0000","last_peered":"2024-04-28T12:49:57.198025+0000","last_clean":"2024-04-28T12:49:57.198025+0000","last_became_active":"2024-04-28T12:48:54.129437+0000","last_became_peered":"2024-04-28T12:48:54.129437+0000","last_unstale":"2024-04-28T12:49:57.198025+0000","last_undegraded":"2024-04-28T12:49:57.198025+0000","last_fullsized":"2024-04-28T12:49:57.198025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:22:14.658847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202561+0000","last_change":"2024-04-28T12:48:54.980482+0000","last_active":"2024-04-28T12:49:57.202561+0000","last_peered":"2024-04-28T12:49:57.202561+0000","last_clean":"2024-04-28T12:49:57.202561+0000","last_became_active":"2024-04-28T12:48:54.980022+0000","last_became_peered":"2024-04-28T12:48:54.980022+0000","last_unstale":"2024-04-28T12:49:57.202561+0000","last_undegraded":"2024-04-28T12:49:57.202561+0000","last_fullsized":"2024-04-28T12:49:57.202561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:50:11.573598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.211378+0000","last_change":"2024-04-28T12:48:59.188749+0000","last_active":"2024-04-28T12:49:57.211378+0000","last_peered":"2024-04-28T12:49:57.211378+0000","last_clean":"2024-04-28T12:49:57.211378+0000","last_became_active":"2024-04-28T12:48:59.187735+0000","last_became_peered":"2024-04-28T12:48:59.187735+0000","last_unstale":"2024-04-28T12:49:57.211378+0000","last_undegraded":"2024-04-28T12:49:57.211378+0000","last_fullsized":"2024-04-28T12:49:57.211378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:50:42.948378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.437935+0000","last_change":"2024-04-28T12:48:54.132690+0000","last_active":"2024-04-28T12:49:57.437935+0000","last_peered":"2024-04-28T12:49:57.437935+0000","last_clean":"2024-04-28T12:49:57.437935+0000","last_became_active":"2024-04-28T12:48:54.132300+0000","last_became_peered":"2024-04-28T12:48:54.132300+0000","last_unstale":"2024-04-28T12:49:57.437935+0000","last_undegraded":"2024-04-28T12:49:57.437935+0000","last_fullsized":"2024-04-28T12:49:57.437935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:25:14.194355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.212186+0000","last_change":"2024-04-28T12:48:54.981277+0000","last_active":"2024-04-28T12:49:57.212186+0000","last_peered":"2024-04-28T12:49:57.212186+0000","last_clean":"2024-04-28T12:49:57.212186+0000","last_became_active":"2024-04-28T12:48:54.977366+0000","last_became_peered":"2024-04-28T12:48:54.977366+0000","last_unstale":"2024-04-28T12:49:57.212186+0000","last_undegraded":"2024-04-28T12:49:57.212186+0000","last_fullsized":"2024-04-28T12:49:57.212186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:12:25.323001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193311+0000","last_change":"2024-04-28T12:48:59.174010+0000","last_active":"2024-04-28T12:49:57.193311+0000","last_peered":"2024-04-28T12:49:57.193311+0000","last_clean":"2024-04-28T12:49:57.193311+0000","last_became_active":"2024-04-28T12:48:59.173370+0000","last_became_peered":"2024-04-28T12:48:59.173370+0000","last_unstale":"2024-04-28T12:49:57.193311+0000","last_undegraded":"2024-04-28T12:49:57.193311+0000","last_fullsized":"2024-04-28T12:49:57.193311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:07:43.258318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193814+0000","last_change":"2024-04-28T12:48:54.127134+0000","last_active":"2024-04-28T12:49:57.193814+0000","last_peered":"2024-04-28T12:49:57.193814+0000","last_clean":"2024-04-28T12:49:57.193814+0000","last_became_active":"2024-04-28T12:48:54.126845+0000","last_became_peered":"2024-04-28T12:48:54.126845+0000","last_unstale":"2024-04-28T12:49:57.193814+0000","last_undegraded":"2024-04-28T12:49:57.193814+0000","last_fullsized":"2024-04-28T12:49:57.193814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:43.502780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.207587+0000","last_change":"2024-04-28T12:49:57.207587+0000","last_active":"2024-04-28T12:49:57.207587+0000","last_peered":"2024-04-28T12:49:57.207587+0000","last_clean":"2024-04-28T12:49:57.207587+0000","last_became_active":"2024-04-28T12:49:57.206688+0000","last_became_peered":"2024-04-28T12:49:57.206688+0000","last_unstale":"2024-04-28T12:49:57.207587+0000","last_undegraded":"2024-04-28T12:49:57.207587+0000","last_fullsized":"2024-04-28T12:49:57.207587+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:15:27.801898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,4],"acting":[0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193881+0000","last_change":"2024-04-28T12:48:59.188456+0000","last_active":"2024-04-28T12:49:57.193881+0000","last_peered":"2024-04-28T12:49:57.193881+0000","last_clean":"2024-04-28T12:49:57.193881+0000","last_became_active":"2024-04-28T12:48:59.187984+0000","last_became_peered":"2024-04-28T12:48:59.187984+0000","last_unstale":"2024-04-28T12:49:57.193881+0000","last_undegraded":"2024-04-28T12:49:57.193881+0000","last_fullsized":"2024-04-28T12:49:57.193881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:38:32.653563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198715+0000","last_change":"2024-04-28T12:48:54.125483+0000","last_active":"2024-04-28T12:49:57.198715+0000","last_peered":"2024-04-28T12:49:57.198715+0000","last_clean":"2024-04-28T12:49:57.198715+0000","last_became_active":"2024-04-28T12:48:54.125261+0000","last_became_peered":"2024-04-28T12:48:54.125261+0000","last_unstale":"2024-04-28T12:49:57.198715+0000","last_undegraded":"2024-04-28T12:49:57.198715+0000","last_fullsized":"2024-04-28T12:49:57.198715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:32:24.986128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197447+0000","last_change":"2024-04-28T12:48:54.960662+0000","last_active":"2024-04-28T12:49:57.197447+0000","last_peered":"2024-04-28T12:49:57.197447+0000","last_clean":"2024-04-28T12:49:57.197447+0000","last_became_active":"2024-04-28T12:48:54.960295+0000","last_became_peered":"2024-04-28T12:48:54.960295+0000","last_unstale":"2024-04-28T12:49:57.197447+0000","last_undegraded":"2024-04-28T12:49:57.197447+0000","last_fullsized":"2024-04-28T12:49:57.197447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:25:26.190901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.438135+0000","last_change":"2024-04-28T12:48:59.194137+0000","last_active":"2024-04-28T12:49:57.438135+0000","last_peered":"2024-04-28T12:49:57.438135+0000","last_clean":"2024-04-28T12:49:57.438135+0000","last_became_active":"2024-04-28T12:48:59.193740+0000","last_became_peered":"2024-04-28T12:48:59.193740+0000","last_unstale":"2024-04-28T12:49:57.438135+0000","last_undegraded":"2024-04-28T12:49:57.438135+0000","last_fullsized":"2024-04-28T12:49:57.438135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:58:52.962196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.208249+0000","last_change":"2024-04-28T12:48:54.140883+0000","last_active":"2024-04-28T12:49:57.208249+0000","last_peered":"2024-04-28T12:49:57.208249+0000","last_clean":"2024-04-28T12:49:57.208249+0000","last_became_active":"2024-04-28T12:48:54.140548+0000","last_became_peered":"2024-04-28T12:48:54.140548+0000","last_unstale":"2024-04-28T12:49:57.208249+0000","last_undegraded":"2024-04-28T12:49:57.208249+0000","last_fullsized":"2024-04-28T12:49:57.208249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:33:31.465266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195119+0000","last_change":"2024-04-28T12:48:54.976359+0000","last_active":"2024-04-28T12:49:57.195119+0000","last_peered":"2024-04-28T12:49:57.195119+0000","last_clean":"2024-04-28T12:49:57.195119+0000","last_became_active":"2024-04-28T12:48:54.976028+0000","last_became_peered":"2024-04-28T12:48:54.976028+0000","last_unstale":"2024-04-28T12:49:57.195119+0000","last_undegraded":"2024-04-28T12:49:57.195119+0000","last_fullsized":"2024-04-28T12:49:57.195119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:02:48.498276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193667+0000","last_change":"2024-04-28T12:48:59.179755+0000","last_active":"2024-04-28T12:49:57.193667+0000","last_peered":"2024-04-28T12:49:57.193667+0000","last_clean":"2024-04-28T12:49:57.193667+0000","last_became_active":"2024-04-28T12:48:59.179258+0000","last_became_peered":"2024-04-28T12:48:59.179258+0000","last_unstale":"2024-04-28T12:49:57.193667+0000","last_undegraded":"2024-04-28T12:49:57.193667+0000","last_fullsized":"2024-04-28T12:49:57.193667+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:59:38.863875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.217857+0000","last_change":"2024-04-28T12:49:57.217468+0000","last_active":"2024-04-28T12:49:57.217857+0000","last_peered":"2024-04-28T12:49:57.217857+0000","last_clean":"2024-04-28T12:49:57.217857+0000","last_became_active":"2024-04-28T12:49:57.212697+0000","last_became_peered":"2024-04-28T12:49:57.212697+0000","last_unstale":"2024-04-28T12:49:57.217857+0000","last_undegraded":"2024-04-28T12:49:57.217857+0000","last_fullsized":"2024-04-28T12:49:57.217857+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:57:20.543124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.207772+0000","last_change":"2024-04-28T12:48:54.990809+0000","last_active":"2024-04-28T12:49:57.207772+0000","last_peered":"2024-04-28T12:49:57.207772+0000","last_clean":"2024-04-28T12:49:57.207772+0000","last_became_active":"2024-04-28T12:48:54.990251+0000","last_became_peered":"2024-04-28T12:48:54.990251+0000","last_unstale":"2024-04-28T12:49:57.207772+0000","last_undegraded":"2024-04-28T12:49:57.207772+0000","last_fullsized":"2024-04-28T12:49:57.207772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:14:30.017815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193974+0000","last_change":"2024-04-28T12:48:59.193287+0000","last_active":"2024-04-28T12:49:57.193974+0000","last_peered":"2024-04-28T12:49:57.193974+0000","last_clean":"2024-04-28T12:49:57.193974+0000","last_became_active":"2024-04-28T12:48:59.192755+0000","last_became_peered":"2024-04-28T12:48:59.192755+0000","last_unstale":"2024-04-28T12:49:57.193974+0000","last_undegraded":"2024-04-28T12:49:57.193974+0000","last_fullsized":"2024-04-28T12:49:57.193974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:00:33.790384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.204891+0000","last_change":"2024-04-28T12:49:57.204548+0000","last_active":"2024-04-28T12:49:57.204891+0000","last_peered":"2024-04-28T12:49:57.204891+0000","last_clean":"2024-04-28T12:49:57.204891+0000","last_became_active":"2024-04-28T12:49:57.199078+0000","last_became_peered":"2024-04-28T12:49:57.199078+0000","last_unstale":"2024-04-28T12:49:57.204891+0000","last_undegraded":"2024-04-28T12:49:57.204891+0000","last_fullsized":"2024-04-28T12:49:57.204891+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:21:46.153453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197870+0000","last_change":"2024-04-28T12:48:54.970270+0000","last_active":"2024-04-28T12:49:57.197870+0000","last_peered":"2024-04-28T12:49:57.197870+0000","last_clean":"2024-04-28T12:49:57.197870+0000","last_became_active":"2024-04-28T12:48:54.970010+0000","last_became_peered":"2024-04-28T12:48:54.970010+0000","last_unstale":"2024-04-28T12:49:57.197870+0000","last_undegraded":"2024-04-28T12:49:57.197870+0000","last_fullsized":"2024-04-28T12:49:57.197870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-30T00:05:03.578751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198646+0000","last_change":"2024-04-28T12:48:59.186197+0000","last_active":"2024-04-28T12:49:57.198646+0000","last_peered":"2024-04-28T12:49:57.198646+0000","last_clean":"2024-04-28T12:49:57.198646+0000","last_became_active":"2024-04-28T12:48:59.185749+0000","last_became_peered":"2024-04-28T12:48:59.185749+0000","last_unstale":"2024-04-28T12:49:57.198646+0000","last_undegraded":"2024-04-28T12:49:57.198646+0000","last_fullsized":"2024-04-28T12:49:57.198646+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:13:36.336775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":116,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:50:49.210228+0000","last_change":"2024-04-28T12:48:54.118725+0000","last_active":"2024-04-28T12:50:49.210228+0000","last_peered":"2024-04-28T12:50:49.210228+0000","last_clean":"2024-04-28T12:50:49.210228+0000","last_became_active":"2024-04-28T12:48:54.118419+0000","last_became_peered":"2024-04-28T12:48:54.118419+0000","last_unstale":"2024-04-28T12:50:49.210228+0000","last_undegraded":"2024-04-28T12:50:49.210228+0000","last_fullsized":"2024-04-28T12:50:49.210228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:25:17.959905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198356+0000","last_change":"2024-04-28T12:48:54.959518+0000","last_active":"2024-04-28T12:49:57.198356+0000","last_peered":"2024-04-28T12:49:57.198356+0000","last_clean":"2024-04-28T12:49:57.198356+0000","last_became_active":"2024-04-28T12:48:54.959101+0000","last_became_peered":"2024-04-28T12:48:54.959101+0000","last_unstale":"2024-04-28T12:49:57.198356+0000","last_undegraded":"2024-04-28T12:49:57.198356+0000","last_fullsized":"2024-04-28T12:49:57.198356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:38:50.496961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.199527+0000","last_change":"2024-04-28T12:48:59.180941+0000","last_active":"2024-04-28T12:49:57.199527+0000","last_peered":"2024-04-28T12:49:57.199527+0000","last_clean":"2024-04-28T12:49:57.199527+0000","last_became_active":"2024-04-28T12:48:59.176889+0000","last_became_peered":"2024-04-28T12:48:59.176889+0000","last_unstale":"2024-04-28T12:49:57.199527+0000","last_undegraded":"2024-04-28T12:49:57.199527+0000","last_fullsized":"2024-04-28T12:49:57.199527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:28:21.853493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195932+0000","last_change":"2024-04-28T12:48:54.126265+0000","last_active":"2024-04-28T12:49:57.195932+0000","last_peered":"2024-04-28T12:49:57.195932+0000","last_clean":"2024-04-28T12:49:57.195932+0000","last_became_active":"2024-04-28T12:48:54.125918+0000","last_became_peered":"2024-04-28T12:48:54.125918+0000","last_unstale":"2024-04-28T12:49:57.195932+0000","last_undegraded":"2024-04-28T12:49:57.195932+0000","last_fullsized":"2024-04-28T12:49:57.195932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:24:26.340143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197820+0000","last_change":"2024-04-28T12:48:54.985335+0000","last_active":"2024-04-28T12:49:57.197820+0000","last_peered":"2024-04-28T12:49:57.197820+0000","last_clean":"2024-04-28T12:49:57.197820+0000","last_became_active":"2024-04-28T12:48:54.985017+0000","last_became_peered":"2024-04-28T12:48:54.985017+0000","last_unstale":"2024-04-28T12:49:57.197820+0000","last_undegraded":"2024-04-28T12:49:57.197820+0000","last_fullsized":"2024-04-28T12:49:57.197820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T13:49:21.821976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195330+0000","last_change":"2024-04-28T12:48:59.176438+0000","last_active":"2024-04-28T12:49:57.195330+0000","last_peered":"2024-04-28T12:49:57.195330+0000","last_clean":"2024-04-28T12:49:57.195330+0000","last_became_active":"2024-04-28T12:48:59.175773+0000","last_became_peered":"2024-04-28T12:48:59.175773+0000","last_unstale":"2024-04-28T12:49:57.195330+0000","last_undegraded":"2024-04-28T12:49:57.195330+0000","last_fullsized":"2024-04-28T12:49:57.195330+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:58.132236+0000","last_clean_scrub_stamp":"2024-04-28T12:48:58.132236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T14:53:32.687553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202953+0000","last_change":"2024-04-28T12:48:54.130452+0000","last_active":"2024-04-28T12:49:57.202953+0000","last_peered":"2024-04-28T12:49:57.202953+0000","last_clean":"2024-04-28T12:49:57.202953+0000","last_became_active":"2024-04-28T12:48:54.130090+0000","last_became_peered":"2024-04-28T12:48:54.130090+0000","last_unstale":"2024-04-28T12:49:57.202953+0000","last_undegraded":"2024-04-28T12:49:57.202953+0000","last_fullsized":"2024-04-28T12:49:57.202953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:53.078207+0000","last_clean_scrub_stamp":"2024-04-28T12:48:53.078207+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:02:34.014710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198233+0000","last_change":"2024-04-28T12:48:54.961838+0000","last_active":"2024-04-28T12:49:57.198233+0000","last_peered":"2024-04-28T12:49:57.198233+0000","last_clean":"2024-04-28T12:49:57.198233+0000","last_became_active":"2024-04-28T12:48:54.961525+0000","last_became_peered":"2024-04-28T12:48:54.961525+0000","last_unstale":"2024-04-28T12:49:57.198233+0000","last_undegraded":"2024-04-28T12:49:57.198233+0000","last_fullsized":"2024-04-28T12:49:57.198233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:06:46.077023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195168+0000","last_change":"2024-04-28T12:48:54.959806+0000","last_active":"2024-04-28T12:49:57.195168+0000","last_peered":"2024-04-28T12:49:57.195168+0000","last_clean":"2024-04-28T12:49:57.195168+0000","last_became_active":"2024-04-28T12:48:54.959491+0000","last_became_peered":"2024-04-28T12:48:54.959491+0000","last_unstale":"2024-04-28T12:49:57.195168+0000","last_undegraded":"2024-04-28T12:49:57.195168+0000","last_fullsized":"2024-04-28T12:49:57.195168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:12:25.367185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193620+0000","last_change":"2024-04-28T12:48:54.960035+0000","last_active":"2024-04-28T12:49:57.193620+0000","last_peered":"2024-04-28T12:49:57.193620+0000","last_clean":"2024-04-28T12:49:57.193620+0000","last_became_active":"2024-04-28T12:48:54.959684+0000","last_became_peered":"2024-04-28T12:48:54.959684+0000","last_unstale":"2024-04-28T12:49:57.193620+0000","last_undegraded":"2024-04-28T12:49:57.193620+0000","last_fullsized":"2024-04-28T12:49:57.193620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T22:46:36.484928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193393+0000","last_change":"2024-04-28T12:48:54.984682+0000","last_active":"2024-04-28T12:49:57.193393+0000","last_peered":"2024-04-28T12:49:57.193393+0000","last_clean":"2024-04-28T12:49:57.193393+0000","last_became_active":"2024-04-28T12:48:54.984346+0000","last_became_peered":"2024-04-28T12:48:54.984346+0000","last_unstale":"2024-04-28T12:49:57.193393+0000","last_undegraded":"2024-04-28T12:49:57.193393+0000","last_fullsized":"2024-04-28T12:49:57.193393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:22:57.828641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.200399+0000","last_change":"2024-04-28T12:48:54.981526+0000","last_active":"2024-04-28T12:49:57.200399+0000","last_peered":"2024-04-28T12:49:57.200399+0000","last_clean":"2024-04-28T12:49:57.200399+0000","last_became_active":"2024-04-28T12:48:54.981212+0000","last_became_peered":"2024-04-28T12:48:54.981212+0000","last_unstale":"2024-04-28T12:49:57.200399+0000","last_undegraded":"2024-04-28T12:49:57.200399+0000","last_fullsized":"2024-04-28T12:49:57.200399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:29:00.976903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.439882+0000","last_change":"2024-04-28T12:49:57.439882+0000","last_active":"2024-04-28T12:49:57.439882+0000","last_peered":"2024-04-28T12:49:57.439882+0000","last_clean":"2024-04-28T12:49:57.439882+0000","last_became_active":"2024-04-28T12:49:57.439390+0000","last_became_peered":"2024-04-28T12:49:57.439390+0000","last_unstale":"2024-04-28T12:49:57.439882+0000","last_undegraded":"2024-04-28T12:49:57.439882+0000","last_fullsized":"2024-04-28T12:49:57.439882+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:39:01.414195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197213+0000","last_change":"2024-04-28T12:48:54.964342+0000","last_active":"2024-04-28T12:49:57.197213+0000","last_peered":"2024-04-28T12:49:57.197213+0000","last_clean":"2024-04-28T12:49:57.197213+0000","last_became_active":"2024-04-28T12:48:54.963980+0000","last_became_peered":"2024-04-28T12:48:54.963980+0000","last_unstale":"2024-04-28T12:49:57.197213+0000","last_undegraded":"2024-04-28T12:49:57.197213+0000","last_fullsized":"2024-04-28T12:49:57.197213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T20:17:07.772838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.193856+0000","last_change":"2024-04-28T12:48:54.963955+0000","last_active":"2024-04-28T12:49:57.193856+0000","last_peered":"2024-04-28T12:49:57.193856+0000","last_clean":"2024-04-28T12:49:57.193856+0000","last_became_active":"2024-04-28T12:48:54.963572+0000","last_became_peered":"2024-04-28T12:48:54.963572+0000","last_unstale":"2024-04-28T12:49:57.193856+0000","last_undegraded":"2024-04-28T12:49:57.193856+0000","last_fullsized":"2024-04-28T12:49:57.193856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T16:40:18.077455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.212365+0000","last_change":"2024-04-28T12:48:54.978143+0000","last_active":"2024-04-28T12:49:57.212365+0000","last_peered":"2024-04-28T12:49:57.212365+0000","last_clean":"2024-04-28T12:49:57.212365+0000","last_became_active":"2024-04-28T12:48:54.977715+0000","last_became_peered":"2024-04-28T12:48:54.977715+0000","last_unstale":"2024-04-28T12:49:57.212365+0000","last_undegraded":"2024-04-28T12:49:57.212365+0000","last_fullsized":"2024-04-28T12:49:57.212365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T21:47:17.071081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.195728+0000","last_change":"2024-04-28T12:48:54.982613+0000","last_active":"2024-04-28T12:49:57.195728+0000","last_peered":"2024-04-28T12:49:57.195728+0000","last_clean":"2024-04-28T12:49:57.195728+0000","last_became_active":"2024-04-28T12:48:54.982231+0000","last_became_peered":"2024-04-28T12:48:54.982231+0000","last_unstale":"2024-04-28T12:49:57.195728+0000","last_undegraded":"2024-04-28T12:49:57.195728+0000","last_fullsized":"2024-04-28T12:49:57.195728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T23:18:22.206377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.208447+0000","last_change":"2024-04-28T12:48:54.978398+0000","last_active":"2024-04-28T12:49:57.208447+0000","last_peered":"2024-04-28T12:49:57.208447+0000","last_clean":"2024-04-28T12:49:57.208447+0000","last_became_active":"2024-04-28T12:48:54.977015+0000","last_became_peered":"2024-04-28T12:48:54.977015+0000","last_unstale":"2024-04-28T12:49:57.208447+0000","last_undegraded":"2024-04-28T12:49:57.208447+0000","last_fullsized":"2024-04-28T12:49:57.208447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T18:39:39.499850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.198856+0000","last_change":"2024-04-28T12:48:54.982870+0000","last_active":"2024-04-28T12:49:57.198856+0000","last_peered":"2024-04-28T12:49:57.198856+0000","last_clean":"2024-04-28T12:49:57.198856+0000","last_became_active":"2024-04-28T12:48:54.982308+0000","last_became_peered":"2024-04-28T12:48:54.982308+0000","last_unstale":"2024-04-28T12:49:57.198856+0000","last_undegraded":"2024-04-28T12:49:57.198856+0000","last_fullsized":"2024-04-28T12:49:57.198856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T17:47:45.976825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.202230+0000","last_change":"2024-04-28T12:48:54.987904+0000","last_active":"2024-04-28T12:49:57.202230+0000","last_peered":"2024-04-28T12:49:57.202230+0000","last_clean":"2024-04-28T12:49:57.202230+0000","last_became_active":"2024-04-28T12:48:54.987551+0000","last_became_peered":"2024-04-28T12:48:54.987551+0000","last_unstale":"2024-04-28T12:49:57.202230+0000","last_undegraded":"2024-04-28T12:49:57.202230+0000","last_fullsized":"2024-04-28T12:49:57.202230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T19:30:36.831005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-28T12:49:57.197169+0000","last_change":"2024-04-28T12:48:54.963176+0000","last_active":"2024-04-28T12:49:57.197169+0000","last_peered":"2024-04-28T12:49:57.197169+0000","last_clean":"2024-04-28T12:49:57.197169+0000","last_became_active":"2024-04-28T12:48:54.962830+0000","last_became_peered":"2024-04-28T12:48:54.962830+0000","last_unstale":"2024-04-28T12:49:57.197169+0000","last_undegraded":"2024-04-28T12:49:57.197169+0000","last_fullsized":"2024-04-28T12:49:57.197169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-28T12:48:54.087196+0000","last_clean_scrub_stamp":"2024-04-28T12:48:54.087196+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-29T15:10:40.671376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59366,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":837,"num_read_kb":483,"num_write":242,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":178098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":501712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1140299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808879,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":505347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070518,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":501629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332157,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1098517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495681,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":496755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":549556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":498602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1110016,"data_stored":550558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":496463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1040384,"data_stored":501220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36964,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708188,"statfs":{"total":95995035648,"available":95957184512,"internally_reserved":0,"allocated":1658880,"data_stored":1093708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":53523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":53898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-28T12:50:54.735 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-28T12:50:54.735 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-28T12:50:54.735 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-28T12:50:54.735 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-28T12:50:55.224 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:50:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:55 smithi171 ceph-mon[63127]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:50:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:55 smithi171 ceph-mon[63127]: pgmap v380: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:55.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:55 smithi129 ceph-mon[62418]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:50:55.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:55 smithi129 ceph-mon[62418]: pgmap v380: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:55.547 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:55 smithi106 ceph-mon[28081]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-28T12:50:55.547 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:55 smithi106 ceph-mon[28081]: pgmap v380: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:57.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.368+0000 7fa621000700 1 -- 172.21.15.106:0/3124401122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c105640 msgr2=0x7fa61c105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:57.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.368+0000 7fa621000700 1 --2- 172.21.15.106:0/3124401122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c105640 0x7fa61c105ac0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa6080098f0 tx=0x7fa608009c00 comp rx=0 tx=0).stop 2024-04-28T12:50:57.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.368+0000 7fa621000700 1 -- 172.21.15.106:0/3124401122 shutdown_connections 2024-04-28T12:50:57.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.368+0000 7fa621000700 1 --2- 172.21.15.106:0/3124401122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa61c106000 0x7fa61c112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.368+0000 7fa621000700 1 --2- 172.21.15.106:0/3124401122 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa61c10b640 0x7fa61c10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.365 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.368+0000 7fa621000700 1 --2- 172.21.15.106:0/3124401122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c105640 0x7fa61c105ac0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.368+0000 7fa621000700 1 -- 172.21.15.106:0/3124401122 >> 172.21.15.106:0/3124401122 conn(0x7fa61c101130 msgr2=0x7fa61c103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:57.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.368+0000 7fa621000700 1 -- 172.21.15.106:0/3124401122 shutdown_connections 2024-04-28T12:50:57.366 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.368+0000 7fa621000700 1 -- 172.21.15.106:0/3124401122 wait complete. 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.369+0000 7fa621000700 1 Processor -- start 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.369+0000 7fa621000700 1 -- start start 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.369+0000 7fa621000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa61c105640 0x7fa61c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.369+0000 7fa621000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa61c106000 0x7fa61c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa621000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c10b640 0x7fa61c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa621000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c1158d0 con 0x7fa61c105640 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa621000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c115750 con 0x7fa61c10b640 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa621000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c115a50 con 0x7fa61c106000 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa619e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa61c105640 0x7fa61c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa619e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa61c105640 0x7fa61c19bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52972/0 (socket says 172.21.15.106:52972) 2024-04-28T12:50:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa619e00700 1 -- 172.21.15.106:0/356688579 learned_addr learned my addr 172.21.15.106:0/356688579 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa619400700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa61c106000 0x7fa61c19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa61a800700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c10b640 0x7fa61c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa619e00700 1 -- 172.21.15.106:0/356688579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa61c106000 msgr2=0x7fa61c19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa619e00700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa61c106000 0x7fa61c19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa619e00700 1 -- 172.21.15.106:0/356688579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c10b640 msgr2=0x7fa61c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa619e00700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c10b640 0x7fa61c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.370+0000 7fa619e00700 1 -- 172.21.15.106:0/356688579 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c1a0d90 con 0x7fa61c105640 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.371+0000 7fa619e00700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa61c105640 0x7fa61c19bd80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00e9a0 tx=0x7fa60c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.371+0000 7fa612a00700 1 -- 172.21.15.106:0/356688579 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00ca10 con 0x7fa61c105640 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.371+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c1a1080 con 0x7fa61c105640 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.371+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1a8900 con 0x7fa61c105640 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.371+0000 7fa612a00700 1 -- 172.21.15.106:0/356688579 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa60c00cbb0 con 0x7fa61c105640 2024-04-28T12:50:57.369 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.371+0000 7fa612a00700 1 -- 172.21.15.106:0/356688579 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c018680 con 0x7fa61c105640 2024-04-28T12:50:57.370 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.372+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61c06a110 con 0x7fa61c105640 2024-04-28T12:50:57.371 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.375+0000 7fa612a00700 1 -- 172.21.15.106:0/356688579 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa60c010430 con 0x7fa61c105640 2024-04-28T12:50:57.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.376+0000 7fa612a00700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa6000a3d50 0x7fa6000a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:57.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.376+0000 7fa612a00700 1 -- 172.21.15.106:0/356688579 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9216+0+0 (secure 0 0 0) 0x7fa60c0d0d00 con 0x7fa61c105640 2024-04-28T12:50:57.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.376+0000 7fa619400700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa6000a3d50 0x7fa6000a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:57.373 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.377+0000 7fa619400700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa6000a3d50 0x7fa6000a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa6080098c0 tx=0x7fa608019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:57.375 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.379+0000 7fa612a00700 1 -- 172.21.15.106:0/356688579 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa60c09d170 con 0x7fa61c105640 2024-04-28T12:50:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:57 smithi171 ceph-mon[63127]: pgmap v381: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-04-28T12:50:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-04-28T12:50:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-04-28T12:50:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-28T12:50:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-28T12:50:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:57 smithi129 ceph-mon[62418]: pgmap v381: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-04-28T12:50:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-04-28T12:50:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-04-28T12:50:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-28T12:50:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-28T12:50:57.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:57 smithi106 ceph-mon[28081]: pgmap v381: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:50:57.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]: dispatch 2024-04-28T12:50:57.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-04-28T12:50:57.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-04-28T12:50:57.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-28T12:50:57.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-28T12:50:57.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.656+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa61c19d0e0 con 0x7fa61c105640 2024-04-28T12:50:57.654 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.657+0000 7fa612a00700 1 -- 172.21.15.106:0/356688579 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa60c09ca70 con 0x7fa61c105640 2024-04-28T12:50:57.654 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:50:57.654 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-28T12:50:57.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa6000a3d50 msgr2=0x7fa6000a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:57.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa6000a3d50 0x7fa6000a6210 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa6080098c0 tx=0x7fa608019040 comp rx=0 tx=0).stop 2024-04-28T12:50:57.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa61c105640 msgr2=0x7fa61c19bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:57.657 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa61c105640 0x7fa61c19bd80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00e9a0 tx=0x7fa60c00ecb0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 shutdown_connections 2024-04-28T12:50:57.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa61c10b640 0x7fa61c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa61c105640 0x7fa61c19bd80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa6000a3d50 0x7fa6000a6210 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 --2- 172.21.15.106:0/356688579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa61c106000 0x7fa61c19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:57.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.660+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 >> 172.21.15.106:0/356688579 conn(0x7fa61c101130 msgr2=0x7fa61c101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:57.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.661+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 shutdown_connections 2024-04-28T12:50:57.658 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:57.661+0000 7fa621000700 1 -- 172.21.15.106:0/356688579 wait complete. 2024-04-28T12:50:58.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-04-28T12:50:58.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-04-28T12:50:58.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-04-28T12:50:58.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-28T12:50:58.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-28T12:50:58.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:58 smithi106 ceph-mon[28081]: osdmap e92: 12 total, 12 up, 12 in 2024-04-28T12:50:58.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:58 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/356688579' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-28T12:50:58.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:58 smithi106 ceph-mon[28081]: osdmap e93: 12 total, 12 up, 12 in 2024-04-28T12:50:58.476 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-28T12:50:58.477 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:50:58.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-04-28T12:50:58.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-04-28T12:50:58.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-04-28T12:50:58.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-28T12:50:58.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-28T12:50:58.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:58 smithi171 ceph-mon[63127]: osdmap e92: 12 total, 12 up, 12 in 2024-04-28T12:50:58.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:58 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/356688579' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-28T12:50:58.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:58 smithi171 ceph-mon[63127]: osdmap e93: 12 total, 12 up, 12 in 2024-04-28T12:50:58.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.5", "id": [3, 9]}]': finished 2024-04-28T12:50:58.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-04-28T12:50:58.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-04-28T12:50:58.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-28T12:50:58.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-28T12:50:58.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:58 smithi129 ceph-mon[62418]: osdmap e92: 12 total, 12 up, 12 in 2024-04-28T12:50:58.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:58 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/356688579' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-28T12:50:58.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:58 smithi129 ceph-mon[62418]: osdmap e93: 12 total, 12 up, 12 in 2024-04-28T12:50:58.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.688+0000 7fa4d5e00640 1 -- 172.21.15.106:0/3140948687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d010a940 msgr2=0x7fa4d01111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:58.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.688+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/3140948687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d010a940 0x7fa4d01111d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009ca0 tx=0x7fa4c802a8c0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.688+0000 7fa4d5e00640 1 -- 172.21.15.106:0/3140948687 shutdown_connections 2024-04-28T12:50:58.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.688+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/3140948687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d010a940 0x7fa4d01111d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.688+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/3140948687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d0109f80 0x7fa4d010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.688+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/3140948687 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4d0108d90 0x7fa4d01091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.688+0000 7fa4d5e00640 1 -- 172.21.15.106:0/3140948687 >> 172.21.15.106:0/3140948687 conn(0x7fa4d0077d30 msgr2=0x7fa4d007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.688+0000 7fa4d5e00640 1 -- 172.21.15.106:0/3140948687 shutdown_connections 2024-04-28T12:50:58.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.688+0000 7fa4d5e00640 1 -- 172.21.15.106:0/3140948687 wait complete. 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.689+0000 7fa4d5e00640 1 Processor -- start 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.689+0000 7fa4d5e00640 1 -- start start 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4d5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d0108d90 0x7fa4d019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4d5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d0109f80 0x7fa4d019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4d5e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4d010a940 0x7fa4d01a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4d5e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0114910 con 0x7fa4d0109f80 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4d5e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0114790 con 0x7fa4d010a940 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4d5e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa4d0114a90 con 0x7fa4d0108d90 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4cfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d0109f80 0x7fa4d019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4cfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d0109f80 0x7fa4d019f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52990/0 (socket says 172.21.15.106:52990) 2024-04-28T12:50:58.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4cfe00640 1 -- 172.21.15.106:0/702319566 learned_addr learned my addr 172.21.15.106:0/702319566 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4d5400640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4d010a940 0x7fa4d01a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4d4a00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d0108d90 0x7fa4d019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4cfe00640 1 -- 172.21.15.106:0/702319566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d0108d90 msgr2=0x7fa4d019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4cfe00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d0108d90 0x7fa4d019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4cfe00640 1 -- 172.21.15.106:0/702319566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4d010a940 msgr2=0x7fa4d01a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4cfe00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4d010a940 0x7fa4d01a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.690+0000 7fa4cfe00640 1 -- 172.21.15.106:0/702319566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4d01a6b70 con 0x7fa4d0109f80 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.691+0000 7fa4cfe00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d0109f80 0x7fa4d019f490 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00e9f0 tx=0x7fa4bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.691+0000 7fa4cd600640 1 -- 172.21.15.106:0/702319566 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc00cd80 con 0x7fa4d0109f80 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.691+0000 7fa4cd600640 1 -- 172.21.15.106:0/702319566 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4bc0054d0 con 0x7fa4d0109f80 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.691+0000 7fa4cd600640 1 -- 172.21.15.106:0/702319566 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc010690 con 0x7fa4d0109f80 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.691+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4d01a6e60 con 0x7fa4d0109f80 2024-04-28T12:50:58.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.691+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4d01a7320 con 0x7fa4d0109f80 2024-04-28T12:50:58.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.692+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4d01100b0 con 0x7fa4d0109f80 2024-04-28T12:50:58.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.695+0000 7fa4cd600640 1 -- 172.21.15.106:0/702319566 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fa4bc004e90 con 0x7fa4d0109f80 2024-04-28T12:50:58.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.696+0000 7fa4cd600640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa4b80a3cc0 0x7fa4b80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:58.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.696+0000 7fa4cd600640 1 -- 172.21.15.106:0/702319566 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fa4bc0d0d00 con 0x7fa4d0109f80 2024-04-28T12:50:58.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.696+0000 7fa4d4a00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa4b80a3cc0 0x7fa4b80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:58.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.697+0000 7fa4d4a00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa4b80a3cc0 0x7fa4b80a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0109de0 tx=0x7fa4c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:58.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.700+0000 7fa4cd600640 1 -- 172.21.15.106:0/702319566 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fa4bc09d170 con 0x7fa4d0109f80 2024-04-28T12:50:58.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.906+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4d002dcd0 con 0x7fa4d0109f80 2024-04-28T12:50:58.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.910+0000 7fa4cd600640 1 -- 172.21.15.106:0/702319566 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24928 (secure 0 0 0) 0x7fa4bc09ca70 con 0x7fa4d0109f80 2024-04-28T12:50:58.907 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:50:58.908 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":93,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:50:58.139109+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa4b80a3cc0 msgr2=0x7fa4b80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa4b80a3cc0 0x7fa4b80a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0109de0 tx=0x7fa4c4005f90 comp rx=0 tx=0).stop 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d0109f80 msgr2=0x7fa4d019f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d0109f80 0x7fa4d019f490 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00e9f0 tx=0x7fa4bc00edf0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 shutdown_connections 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fa4b80a3cc0 0x7fa4b80a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4d0109f80 0x7fa4d019f490 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4d010a940 0x7fa4d01a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 --2- 172.21.15.106:0/702319566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa4d0108d90 0x7fa4d019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 >> 172.21.15.106:0/702319566 conn(0x7fa4d0077d30 msgr2=0x7fa4d010d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 shutdown_connections 2024-04-28T12:50:58.911 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:58.914+0000 7fa4d5e00640 1 -- 172.21.15.106:0/702319566 wait complete. 2024-04-28T12:50:58.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:48:52.916511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:48:53.528808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:48:57.154563+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:50:58.931 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:50:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.142+0000 7f4724e00640 1 -- 172.21.15.106:0/2353024637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720105da0 msgr2=0x7f4720106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.142+0000 7f4724e00640 1 --2- 172.21.15.106:0/2353024637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720105da0 0x7f4720106220 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f470c009860 tx=0x7f470c02a340 comp rx=0 tx=0).stop 2024-04-28T12:50:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.142+0000 7f4724e00640 1 -- 172.21.15.106:0/2353024637 shutdown_connections 2024-04-28T12:50:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.142+0000 7f4724e00640 1 --2- 172.21.15.106:0/2353024637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4720106760 0x7f472010cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.142+0000 7f4724e00640 1 --2- 172.21.15.106:0/2353024637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720105da0 0x7f4720106220 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.142+0000 7f4724e00640 1 --2- 172.21.15.106:0/2353024637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4720104bb0 0x7f4720104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.142+0000 7f4724e00640 1 -- 172.21.15.106:0/2353024637 >> 172.21.15.106:0/2353024637 conn(0x7f4720100120 msgr2=0x7f4720102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.142+0000 7f4724e00640 1 -- 172.21.15.106:0/2353024637 shutdown_connections 2024-04-28T12:50:59.139 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.142+0000 7f4724e00640 1 -- 172.21.15.106:0/2353024637 wait complete. 2024-04-28T12:50:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.143+0000 7f4724e00640 1 Processor -- start 2024-04-28T12:50:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.143+0000 7f4724e00640 1 -- start start 2024-04-28T12:50:59.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f4724e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720104bb0 0x7f472007a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f4724e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4720105da0 0x7f472007ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f4724e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4720106760 0x7f4720077250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f4724e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f47201106c0 con 0x7f4720106760 2024-04-28T12:50:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f4724e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4720110540 con 0x7f4720104bb0 2024-04-28T12:50:59.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f4724e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4720110840 con 0x7f4720105da0 2024-04-28T12:50:59.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f471de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720104bb0 0x7f472007a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:59.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f471e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4720106760 0x7f4720077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:59.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f471de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720104bb0 0x7f472007a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:60974/0 (socket says 172.21.15.106:60974) 2024-04-28T12:50:59.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f471de00640 1 -- 172.21.15.106:0/3043377063 learned_addr learned my addr 172.21.15.106:0/3043377063 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:59.142 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.144+0000 7f471d400640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4720105da0 0x7f472007ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f471de00640 1 -- 172.21.15.106:0/3043377063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4720105da0 msgr2=0x7f472007ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f471de00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4720105da0 0x7f472007ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f471de00640 1 -- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4720106760 msgr2=0x7f4720077250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f471de00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4720106760 0x7f4720077250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f471de00640 1 -- 172.21.15.106:0/3043377063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4720077b10 con 0x7f4720104bb0 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f471e800640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4720106760 0x7f4720077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f471de00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720104bb0 0x7f472007a6e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f471000e9f0 tx=0x7f471000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f4716a00640 1 -- 172.21.15.106:0/3043377063 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471000ccc0 con 0x7f4720104bb0 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f4716a00640 1 -- 172.21.15.106:0/3043377063 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47100035f0 con 0x7f4720104bb0 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f4716a00640 1 -- 172.21.15.106:0/3043377063 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471000ce60 con 0x7f4720104bb0 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47201a7270 con 0x7f4720104bb0 2024-04-28T12:50:59.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.145+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47201a7620 con 0x7f4720104bb0 2024-04-28T12:50:59.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.146+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4720069ba0 con 0x7f4720104bb0 2024-04-28T12:50:59.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.148+0000 7f4716a00640 1 -- 172.21.15.106:0/3043377063 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4710004000 con 0x7f4720104bb0 2024-04-28T12:50:59.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.150+0000 7f4716a00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f47040a3cc0 0x7f47040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:59.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.150+0000 7f4716a00640 1 -- 172.21.15.106:0/3043377063 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f47100d0cd0 con 0x7f4720104bb0 2024-04-28T12:50:59.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.150+0000 7f471d400640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f47040a3cc0 0x7f47040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:59.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.151+0000 7f471d400640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f47040a3cc0 0x7f47040a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f470c02a930 tx=0x7f470c0095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:59.149 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.152+0000 7f4716a00640 1 -- 172.21.15.106:0/3043377063 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f471009d140 con 0x7f4720104bb0 2024-04-28T12:50:59.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:59 smithi106 ceph-mon[28081]: pgmap v383: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:59.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:50:59 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/702319566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:50:59.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.362+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f472002dad0 con 0x7f4720104bb0 2024-04-28T12:50:59.359 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.363+0000 7f4716a00640 1 -- 172.21.15.106:0/3043377063 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f471009ca40 con 0x7f4720104bb0 2024-04-28T12:50:59.360 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T12:50:59.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.365+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f47040a3cc0 msgr2=0x7f47040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.362 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.365+0000 7f4724e00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f47040a3cc0 0x7f47040a6180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f470c02a930 tx=0x7f470c0095b0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.365+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720104bb0 msgr2=0x7f472007a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.365+0000 7f4724e00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720104bb0 0x7f472007a6e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f471000e9f0 tx=0x7f471000edf0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.366+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 shutdown_connections 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.366+0000 7f4724e00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f47040a3cc0 0x7f47040a6180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.366+0000 7f4724e00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4720106760 0x7f4720077250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.366+0000 7f4724e00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4720105da0 0x7f472007ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.366+0000 7f4724e00640 1 --2- 172.21.15.106:0/3043377063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4720104bb0 0x7f472007a6e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.366+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 >> 172.21.15.106:0/3043377063 conn(0x7f4720100120 msgr2=0x7f4720109190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.366+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 shutdown_connections 2024-04-28T12:50:59.363 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.366+0000 7f4724e00640 1 -- 172.21.15.106:0/3043377063 wait complete. 2024-04-28T12:50:59.381 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:50:59.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:59 smithi171 ceph-mon[63127]: pgmap v383: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:59.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:50:59 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/702319566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:50:59.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:59 smithi129 ceph-mon[62418]: pgmap v383: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:50:59.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:50:59 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/702319566' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:50:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.592+0000 7fd70d800640 1 -- 172.21.15.106:0/2538638994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 msgr2=0x7fd70810a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.592+0000 7fd70d800640 1 --2- 172.21.15.106:0/2538638994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 0x7fd70810a440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009900 tx=0x7fd6f002a400 comp rx=0 tx=0).stop 2024-04-28T12:50:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.592+0000 7fd70d800640 1 -- 172.21.15.106:0/2538638994 shutdown_connections 2024-04-28T12:50:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.592+0000 7fd70d800640 1 --2- 172.21.15.106:0/2538638994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 0x7fd70810a440 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.592+0000 7fd70d800640 1 --2- 172.21.15.106:0/2538638994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd70810a980 0x7fd7081112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.592+0000 7fd70d800640 1 --2- 172.21.15.106:0/2538638994 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd708108e70 0x7fd708109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.592+0000 7fd70d800640 1 -- 172.21.15.106:0/2538638994 >> 172.21.15.106:0/2538638994 conn(0x7fd708077e00 msgr2=0x7fd70807a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.593+0000 7fd70d800640 1 -- 172.21.15.106:0/2538638994 shutdown_connections 2024-04-28T12:50:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.593+0000 7fd70d800640 1 -- 172.21.15.106:0/2538638994 wait complete. 2024-04-28T12:50:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.593+0000 7fd70d800640 1 Processor -- start 2024-04-28T12:50:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd70d800640 1 -- start start 2024-04-28T12:50:59.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd70d800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd708108e70 0x7fd70819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd70d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 0x7fd70819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd70d800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd70810a980 0x7fd7081a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd70d800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd708114a20 con 0x7fd708109fc0 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd70d800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd7081148a0 con 0x7fd708108e70 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd70d800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd708114ba0 con 0x7fd70810a980 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd6ffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 0x7fd70819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd6ffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 0x7fd70819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36018/0 (socket says 172.21.15.106:36018) 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd6ffe00640 1 -- 172.21.15.106:0/1474433247 learned_addr learned my addr 172.21.15.106:0/1474433247 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd70ce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd70810a980 0x7fd7081a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.594+0000 7fd707e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd708108e70 0x7fd70819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.595+0000 7fd6ffe00640 1 -- 172.21.15.106:0/1474433247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd70810a980 msgr2=0x7fd7081a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.595+0000 7fd6ffe00640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd70810a980 0x7fd7081a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.595+0000 7fd6ffe00640 1 -- 172.21.15.106:0/1474433247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd708108e70 msgr2=0x7fd70819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.595+0000 7fd6ffe00640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd708108e70 0x7fd70819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.595+0000 7fd6ffe00640 1 -- 172.21.15.106:0/1474433247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7081a6c90 con 0x7fd708109fc0 2024-04-28T12:50:59.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.595+0000 7fd6ffe00640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 0x7fd70819f5b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0031500 tx=0x7fd6f0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:59.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.596+0000 7fd705600640 1 -- 172.21.15.106:0/1474433247 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f003f070 con 0x7fd708109fc0 2024-04-28T12:50:59.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.596+0000 7fd705600640 1 -- 172.21.15.106:0/1474433247 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6f0003e70 con 0x7fd708109fc0 2024-04-28T12:50:59.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.596+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7081a6f20 con 0x7fd708109fc0 2024-04-28T12:50:59.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.596+0000 7fd705600640 1 -- 172.21.15.106:0/1474433247 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0035030 con 0x7fd708109fc0 2024-04-28T12:50:59.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.596+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7081a7320 con 0x7fd708109fc0 2024-04-28T12:50:59.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.597+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd708110180 con 0x7fd708109fc0 2024-04-28T12:50:59.595 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.599+0000 7fd705600640 1 -- 172.21.15.106:0/1474433247 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd6f0042070 con 0x7fd708109fc0 2024-04-28T12:50:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.600+0000 7fd705600640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6ec0a3c40 0x7fd6ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:50:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.601+0000 7fd705600640 1 -- 172.21.15.106:0/1474433247 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fd6f00f16d0 con 0x7fd708109fc0 2024-04-28T12:50:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.601+0000 7fd707e00640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6ec0a3c40 0x7fd6ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:50:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.601+0000 7fd707e00640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6ec0a3c40 0x7fd6ec0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8005d00 tx=0x7fd6f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:50:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.603+0000 7fd705600640 1 -- 172.21.15.106:0/1474433247 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd6f00bd4f0 con 0x7fd708109fc0 2024-04-28T12:50:59.802 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.805+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd70802dad0 con 0x7fd708109fc0 2024-04-28T12:50:59.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.806+0000 7fd705600640 1 -- 172.21.15.106:0/1474433247 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fd6f00c17f0 con 0x7fd708109fc0 2024-04-28T12:50:59.803 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:50:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.808+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6ec0a3c40 msgr2=0x7fd6ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.808+0000 7fd70d800640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6ec0a3c40 0x7fd6ec0a6100 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8005d00 tx=0x7fd6f800b410 comp rx=0 tx=0).stop 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.808+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 msgr2=0x7fd70819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.808+0000 7fd70d800640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 0x7fd70819f5b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0031500 tx=0x7fd6f0003c80 comp rx=0 tx=0).stop 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.809+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 shutdown_connections 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.809+0000 7fd70d800640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd6ec0a3c40 0x7fd6ec0a6100 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.809+0000 7fd70d800640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd708109fc0 0x7fd70819f5b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.809+0000 7fd70d800640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd70810a980 0x7fd7081a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.809+0000 7fd70d800640 1 --2- 172.21.15.106:0/1474433247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd708108e70 0x7fd70819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.809+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 >> 172.21.15.106:0/1474433247 conn(0x7fd708077e00 msgr2=0x7fd70810d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.809+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 shutdown_connections 2024-04-28T12:50:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:50:59.809+0000 7fd70d800640 1 -- 172.21.15.106:0/1474433247 wait complete. 2024-04-28T12:50:59.825 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:51:00.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.033+0000 7f2461400640 1 -- 172.21.15.106:0/380252128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c1033d0 msgr2=0x7f245c1057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.033+0000 7f2461400640 1 --2- 172.21.15.106:0/380252128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c1033d0 0x7f245c1057c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f244c009900 tx=0x7f244c02a410 comp rx=0 tx=0).stop 2024-04-28T12:51:00.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.033+0000 7f2461400640 1 -- 172.21.15.106:0/380252128 shutdown_connections 2024-04-28T12:51:00.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.033+0000 7f2461400640 1 --2- 172.21.15.106:0/380252128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f245c105d00 0x7f245c1080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.033+0000 7f2461400640 1 --2- 172.21.15.106:0/380252128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f245c108630 0x7f245c10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.033+0000 7f2461400640 1 --2- 172.21.15.106:0/380252128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c1033d0 0x7f245c1057c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.033+0000 7f2461400640 1 -- 172.21.15.106:0/380252128 >> 172.21.15.106:0/380252128 conn(0x7f245c0fce00 msgr2=0x7f245c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:00.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.034+0000 7f2461400640 1 -- 172.21.15.106:0/380252128 shutdown_connections 2024-04-28T12:51:00.031 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.034+0000 7f2461400640 1 -- 172.21.15.106:0/380252128 wait complete. 2024-04-28T12:51:00.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.035+0000 7f2461400640 1 Processor -- start 2024-04-28T12:51:00.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.035+0000 7f2461400640 1 -- start start 2024-04-28T12:51:00.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.035+0000 7f2461400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f245c1033d0 0x7f245c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.035+0000 7f2461400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c105d00 0x7f245c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.035+0000 7f2461400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f245c108630 0x7f245c1a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.035+0000 7f2461400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f245c10e4d0 con 0x7f245c108630 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.035+0000 7f2461400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f245c10e350 con 0x7f245c105d00 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.035+0000 7f2461400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f245c10e650 con 0x7f245c1033d0 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f245c108630 0x7f245c1a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f245c1033d0 0x7f245c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f245c1033d0 0x7f245c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:47662/0 (socket says 172.21.15.106:47662) 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f2459e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c105d00 0x7f245c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245a800640 1 -- 172.21.15.106:0/293633588 learned_addr learned my addr 172.21.15.106:0/293633588 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245a800640 1 -- 172.21.15.106:0/293633588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c105d00 msgr2=0x7f245c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245a800640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c105d00 0x7f245c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245a800640 1 -- 172.21.15.106:0/293633588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f245c108630 msgr2=0x7f245c1a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245a800640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f245c108630 0x7f245c1a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245a800640 1 -- 172.21.15.106:0/293633588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c1a2a10 con 0x7f245c1033d0 2024-04-28T12:51:00.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f245a800640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f245c1033d0 0x7f245c19adf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f244c005ab0 tx=0x7f244c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:00.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.036+0000 7f2453400640 1 -- 172.21.15.106:0/293633588 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c03f070 con 0x7f245c1033d0 2024-04-28T12:51:00.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.037+0000 7f2453400640 1 -- 172.21.15.106:0/293633588 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f244c031c50 con 0x7f245c1033d0 2024-04-28T12:51:00.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.037+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c1a2ca0 con 0x7f245c1033d0 2024-04-28T12:51:00.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.037+0000 7f2453400640 1 -- 172.21.15.106:0/293633588 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c035030 con 0x7f245c1033d0 2024-04-28T12:51:00.035 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.037+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c1a3190 con 0x7f245c1033d0 2024-04-28T12:51:00.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.038+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c10a180 con 0x7f245c1033d0 2024-04-28T12:51:00.036 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.040+0000 7f2453400640 1 -- 172.21.15.106:0/293633588 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f244c042070 con 0x7f245c1033d0 2024-04-28T12:51:00.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.041+0000 7f2453400640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f24400a3c40 0x7f24400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.041+0000 7f2453400640 1 -- 172.21.15.106:0/293633588 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f244c0f1790 con 0x7f245c1033d0 2024-04-28T12:51:00.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.041+0000 7f2459e00640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f24400a3c40 0x7f24400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.042+0000 7f2459e00640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f24400a3c40 0x7f24400a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f245c19c300 tx=0x7f2448005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:00.040 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.044+0000 7f2453400640 1 -- 172.21.15.106:0/293633588 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f244c0bd5b0 con 0x7f245c1033d0 2024-04-28T12:51:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.250+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f245c02dad0 con 0x7f245c1033d0 2024-04-28T12:51:00.247 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.251+0000 7f2453400640 1 -- 172.21.15.106:0/293633588 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f244c0c18b0 con 0x7f245c1033d0 2024-04-28T12:51:00.248 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-28T12:51:00.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f24400a3c40 msgr2=0x7f24400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f24400a3c40 0x7f24400a6100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f245c19c300 tx=0x7f2448005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:00.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f245c1033d0 msgr2=0x7f245c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f245c1033d0 0x7f245c19adf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f244c005ab0 tx=0x7f244c003b90 comp rx=0 tx=0).stop 2024-04-28T12:51:00.250 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 shutdown_connections 2024-04-28T12:51:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f24400a3c40 0x7f24400a6100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f245c108630 0x7f245c1a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f245c105d00 0x7f245c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 --2- 172.21.15.106:0/293633588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f245c1033d0 0x7f245c19adf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.253+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 >> 172.21.15.106:0/293633588 conn(0x7f245c0fce00 msgr2=0x7f245c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.254+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 shutdown_connections 2024-04-28T12:51:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.254+0000 7f2461400640 1 -- 172.21.15.106:0/293633588 wait complete. 2024-04-28T12:51:00.269 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:51:00.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.467+0000 7fad65800640 1 -- 172.21.15.106:0/1586629965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad6010a980 msgr2=0x7fad601112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.467+0000 7fad65800640 1 --2- 172.21.15.106:0/1586629965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad6010a980 0x7fad601112a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad58009be0 tx=0x7fad5802a6c0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.467+0000 7fad65800640 1 -- 172.21.15.106:0/1586629965 shutdown_connections 2024-04-28T12:51:00.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.467+0000 7fad65800640 1 --2- 172.21.15.106:0/1586629965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad6010a980 0x7fad601112a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.467+0000 7fad65800640 1 --2- 172.21.15.106:0/1586629965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad60109fc0 0x7fad6010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.467+0000 7fad65800640 1 --2- 172.21.15.106:0/1586629965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad60108e70 0x7fad60109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.467+0000 7fad65800640 1 -- 172.21.15.106:0/1586629965 >> 172.21.15.106:0/1586629965 conn(0x7fad60077e00 msgr2=0x7fad6007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:00.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.467+0000 7fad65800640 1 -- 172.21.15.106:0/1586629965 shutdown_connections 2024-04-28T12:51:00.464 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.467+0000 7fad65800640 1 -- 172.21.15.106:0/1586629965 wait complete. 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.468+0000 7fad65800640 1 Processor -- start 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.468+0000 7fad65800640 1 -- start start 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.468+0000 7fad65800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad60108e70 0x7fad6019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.468+0000 7fad65800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad60109fc0 0x7fad6019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.468+0000 7fad65800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad6010a980 0x7fad601a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad65800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fad601149c0 con 0x7fad60109fc0 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad65800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad60114840 con 0x7fad60108e70 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad65800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fad60114b40 con 0x7fad6010a980 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad57e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad60109fc0 0x7fad6019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad5fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad60108e70 0x7fad6019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad57e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad60109fc0 0x7fad6019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36044/0 (socket says 172.21.15.106:36044) 2024-04-28T12:51:00.467 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad57e00640 1 -- 172.21.15.106:0/1329039064 learned_addr learned my addr 172.21.15.106:0/1329039064 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad64e00640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad6010a980 0x7fad601a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad57e00640 1 -- 172.21.15.106:0/1329039064 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad6010a980 msgr2=0x7fad601a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad57e00640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad6010a980 0x7fad601a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad57e00640 1 -- 172.21.15.106:0/1329039064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad60108e70 msgr2=0x7fad6019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad57e00640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad60108e70 0x7fad6019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.469+0000 7fad57e00640 1 -- 172.21.15.106:0/1329039064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad601a6c20 con 0x7fad60109fc0 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.470+0000 7fad57e00640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad60109fc0 0x7fad6019f540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fad4800e9a0 tx=0x7fad4800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.470+0000 7fad5d600640 1 -- 172.21.15.106:0/1329039064 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4800cd80 con 0x7fad60109fc0 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.470+0000 7fad5d600640 1 -- 172.21.15.106:0/1329039064 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad480108e0 con 0x7fad60109fc0 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.470+0000 7fad5d600640 1 -- 172.21.15.106:0/1329039064 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad48018960 con 0x7fad60109fc0 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.470+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad601a6f10 con 0x7fad60109fc0 2024-04-28T12:51:00.468 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.470+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad601a73d0 con 0x7fad60109fc0 2024-04-28T12:51:00.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.471+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad60110180 con 0x7fad60109fc0 2024-04-28T12:51:00.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.473+0000 7fad5d600640 1 -- 172.21.15.106:0/1329039064 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fad48010a90 con 0x7fad60109fc0 2024-04-28T12:51:00.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.475+0000 7fad5d600640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fad440a3c40 0x7fad440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.476+0000 7fad5d600640 1 -- 172.21.15.106:0/1329039064 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fad480d0cb0 con 0x7fad60109fc0 2024-04-28T12:51:00.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.476+0000 7fad5fe00640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fad440a3c40 0x7fad440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.476+0000 7fad5fe00640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fad440a3c40 0x7fad440a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad50009bd0 tx=0x7fad50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:00.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.480+0000 7fad5d600640 1 -- 172.21.15.106:0/1329039064 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fad48054060 con 0x7fad60109fc0 2024-04-28T12:51:00.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:00 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3043377063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:00.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:00 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1474433247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:00.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:00 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3043377063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:00.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:00 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1474433247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:00.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:00 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3043377063' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:00.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:00 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1474433247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:00.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.681+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fad6002dd20 con 0x7fad60109fc0 2024-04-28T12:51:00.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.682+0000 7fad5d600640 1 -- 172.21.15.106:0/1329039064 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fad4809d120 con 0x7fad60109fc0 2024-04-28T12:51:00.679 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:51:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.684+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fad440a3c40 msgr2=0x7fad440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fad440a3c40 0x7fad440a6100 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad50009bd0 tx=0x7fad50005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad60109fc0 msgr2=0x7fad6019f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad60109fc0 0x7fad6019f540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fad4800e9a0 tx=0x7fad4800ee60 comp rx=0 tx=0).stop 2024-04-28T12:51:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 shutdown_connections 2024-04-28T12:51:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fad440a3c40 0x7fad440a6100 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad60109fc0 0x7fad6019f540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad6010a980 0x7fad601a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 --2- 172.21.15.106:0/1329039064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad60108e70 0x7fad6019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 >> 172.21.15.106:0/1329039064 conn(0x7fad60077e00 msgr2=0x7fad6010d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 shutdown_connections 2024-04-28T12:51:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.685+0000 7fad65800640 1 -- 172.21.15.106:0/1329039064 wait complete. 2024-04-28T12:51:00.701 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:51:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.901+0000 7f573e200640 1 -- 172.21.15.106:0/841972845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 msgr2=0x7f5738106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.901+0000 7f573e200640 1 --2- 172.21.15.106:0/841972845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 0x7f5738106000 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5720009900 tx=0x7f572002a410 comp rx=0 tx=0).stop 2024-04-28T12:51:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.902+0000 7f573e200640 1 -- 172.21.15.106:0/841972845 shutdown_connections 2024-04-28T12:51:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.902+0000 7f573e200640 1 --2- 172.21.15.106:0/841972845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5738106540 0x7f573810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.902+0000 7f573e200640 1 --2- 172.21.15.106:0/841972845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 0x7f5738106000 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.902+0000 7f573e200640 1 --2- 172.21.15.106:0/841972845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738104990 0x7f5738104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.902+0000 7f573e200640 1 -- 172.21.15.106:0/841972845 >> 172.21.15.106:0/841972845 conn(0x7f57380fff00 msgr2=0x7f5738102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.902+0000 7f573e200640 1 -- 172.21.15.106:0/841972845 shutdown_connections 2024-04-28T12:51:00.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.902+0000 7f573e200640 1 -- 172.21.15.106:0/841972845 wait complete. 2024-04-28T12:51:00.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573e200640 1 Processor -- start 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573e200640 1 -- start start 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573e200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5738104990 0x7f573819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573e200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 0x7f573819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573e200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738106540 0x7f57381a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573e200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57381104a0 con 0x7f5738106540 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573e200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5738110320 con 0x7f5738104990 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573e200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5738110620 con 0x7f5738105b80 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738106540 0x7f57381a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f5737e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 0x7f573819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f5737e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 0x7f573819b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:47698/0 (socket says 172.21.15.106:47698) 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f573d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738106540 0x7f57381a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36074/0 (socket says 172.21.15.106:36074) 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.903+0000 7f5737e00640 1 -- 172.21.15.106:0/2726532320 learned_addr learned my addr 172.21.15.106:0/2726532320 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.904+0000 7f573ce00640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5738104990 0x7f573819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.904+0000 7f5737e00640 1 -- 172.21.15.106:0/2726532320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5738104990 msgr2=0x7f573819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.904+0000 7f5737e00640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5738104990 0x7f573819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.904+0000 7f5737e00640 1 -- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738106540 msgr2=0x7f57381a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.904+0000 7f5737e00640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738106540 0x7f57381a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.904+0000 7f5737e00640 1 -- 172.21.15.106:0/2726532320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57381a2770 con 0x7f5738105b80 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.904+0000 7f573d800640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738106540 0x7f57381a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.904+0000 7f5737e00640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 0x7f573819b090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5720005940 tx=0x7f57200042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.904+0000 7f5735600640 1 -- 172.21.15.106:0/2726532320 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f572003f030 con 0x7f5738105b80 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.905+0000 7f5735600640 1 -- 172.21.15.106:0/2726532320 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5720003cb0 con 0x7f5738105b80 2024-04-28T12:51:00.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.905+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57381a2a00 con 0x7f5738105b80 2024-04-28T12:51:00.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.905+0000 7f5735600640 1 -- 172.21.15.106:0/2726532320 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5720035070 con 0x7f5738105b80 2024-04-28T12:51:00.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.905+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57381a2ef0 con 0x7f5738105b80 2024-04-28T12:51:00.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.906+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573810bd20 con 0x7f5738105b80 2024-04-28T12:51:00.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.908+0000 7f5735600640 1 -- 172.21.15.106:0/2726532320 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5720042030 con 0x7f5738105b80 2024-04-28T12:51:00.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.909+0000 7f5735600640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f571c0a3c40 0x7f571c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:00.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.909+0000 7f5735600640 1 -- 172.21.15.106:0/2726532320 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f57200f13e0 con 0x7f5738105b80 2024-04-28T12:51:00.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.909+0000 7f573ce00640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f571c0a3c40 0x7f571c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:00.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.910+0000 7f573ce00640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f571c0a3c40 0x7f571c0a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f57381059e0 tx=0x7f5728009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:00.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:00.912+0000 7f5735600640 1 -- 172.21.15.106:0/2726532320 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5720003e60 con 0x7f5738105b80 2024-04-28T12:51:01.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.158+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f573802dad0 con 0x7f5738105b80 2024-04-28T12:51:01.156 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.160+0000 7f5735600640 1 -- 172.21.15.106:0/2726532320 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11953 (secure 0 0 0) 0x7f57200bd200 con 0x7f5738105b80 2024-04-28T12:51:01.157 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:01.157 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:49:30.023676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24548,"mds_3":24538,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24548,24520,24578,24538,24566]},"id":1}]} 2024-04-28T12:51:01.157 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-28T12:51:01.159 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f571c0a3c40 msgr2=0x7f571c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f571c0a3c40 0x7f571c0a6100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f57381059e0 tx=0x7f5728009500 comp rx=0 tx=0).stop 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 msgr2=0x7f573819b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 0x7f573819b090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5720005940 tx=0x7f57200042f0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 shutdown_connections 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f571c0a3c40 0x7f571c0a6100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5738106540 0x7f57381a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5738105b80 0x7f573819b090 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 --2- 172.21.15.106:0/2726532320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5738104990 0x7f573819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 >> 172.21.15.106:0/2726532320 conn(0x7f57380fff00 msgr2=0x7f5738108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 shutdown_connections 2024-04-28T12:51:01.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.163+0000 7f573e200640 1 -- 172.21.15.106:0/2726532320 wait complete. 2024-04-28T12:51:01.179 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:51:01.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.382+0000 7f1e03200640 1 -- 172.21.15.106:0/2823098282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04105b50 msgr2=0x7f1e04105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.379 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.382+0000 7f1e03200640 1 --2- 172.21.15.106:0/2823098282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04105b50 0x7f1e04105fd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009900 tx=0x7f1dec02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.382+0000 7f1e03200640 1 -- 172.21.15.106:0/2823098282 shutdown_connections 2024-04-28T12:51:01.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.382+0000 7f1e03200640 1 --2- 172.21.15.106:0/2823098282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04105b50 0x7f1e04105fd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.382+0000 7f1e03200640 1 --2- 172.21.15.106:0/2823098282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e04106510 0x7f1e0410cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.382+0000 7f1e03200640 1 --2- 172.21.15.106:0/2823098282 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04104960 0x7f1e04104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.382+0000 7f1e03200640 1 -- 172.21.15.106:0/2823098282 >> 172.21.15.106:0/2823098282 conn(0x7f1e040ffeb0 msgr2=0x7f1e04102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:01.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.382+0000 7f1e03200640 1 -- 172.21.15.106:0/2823098282 shutdown_connections 2024-04-28T12:51:01.380 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.382+0000 7f1e03200640 1 -- 172.21.15.106:0/2823098282 wait complete. 2024-04-28T12:51:01.382 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.383+0000 7f1e03200640 1 Processor -- start 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.383+0000 7f1e03200640 1 -- start start 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e03200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04104960 0x7f1e0419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e03200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04105b50 0x7f1e0419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e03200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e04106510 0x7f1e041a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e03200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e041104e0 con 0x7f1e04104960 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e03200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04110360 con 0x7f1e04106510 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e03200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1e04110660 con 0x7f1e04105b50 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e01e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04104960 0x7f1e0419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e01e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04104960 0x7f1e0419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36090/0 (socket says 172.21.15.106:36090) 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e01e00640 1 -- 172.21.15.106:0/3585476330 learned_addr learned my addr 172.21.15.106:0/3585476330 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1e02800640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e04106510 0x7f1e041a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.384+0000 7f1dfbe00640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04105b50 0x7f1e0419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1e01e00640 1 -- 172.21.15.106:0/3585476330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04105b50 msgr2=0x7f1e0419b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1e01e00640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04105b50 0x7f1e0419b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1e01e00640 1 -- 172.21.15.106:0/3585476330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e04106510 msgr2=0x7f1e041a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1e01e00640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e04106510 0x7f1e041a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1e01e00640 1 -- 172.21.15.106:0/3585476330 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e041a27a0 con 0x7f1e04104960 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1e02800640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e04106510 0x7f1e041a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1e01e00640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04104960 0x7f1e0419abd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1df400e9c0 tx=0x7f1df400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1dfaa00640 1 -- 172.21.15.106:0/3585476330 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df400cd80 con 0x7f1e04104960 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1dfaa00640 1 -- 172.21.15.106:0/3585476330 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df40035f0 con 0x7f1e04104960 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e041a2a90 con 0x7f1e04104960 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1dfaa00640 1 -- 172.21.15.106:0/3585476330 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4010690 con 0x7f1e04104960 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.385+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e041a2f60 con 0x7f1e04104960 2024-04-28T12:51:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.387+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e0410bc80 con 0x7f1e04104960 2024-04-28T12:51:01.385 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.389+0000 7f1dfaa00640 1 -- 172.21.15.106:0/3585476330 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1df4004000 con 0x7f1e04104960 2024-04-28T12:51:01.386 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.390+0000 7f1dfaa00640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1de80a3c40 0x7f1de80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:01.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.390+0000 7f1dfaa00640 1 -- 172.21.15.106:0/3585476330 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f1df40d0e80 con 0x7f1e04104960 2024-04-28T12:51:01.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.390+0000 7f1dfbe00640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1de80a3c40 0x7f1de80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:01.387 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.391+0000 7f1dfbe00640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1de80a3c40 0x7f1de80a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1dec031500 tx=0x7f1dec005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:01.391 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.395+0000 7f1dfaa00640 1 -- 172.21.15.106:0/3585476330 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1df409d2f0 con 0x7f1e04104960 2024-04-28T12:51:01.495 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:01 smithi171 ceph-mon[63127]: pgmap v385: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 127 B/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-04-28T12:51:01.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:01 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/293633588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:01.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:01 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1329039064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:01.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:01 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2726532320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:01.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:01 smithi129 ceph-mon[62418]: pgmap v385: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 127 B/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-04-28T12:51:01.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:01 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/293633588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:01.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:01 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1329039064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:01.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:01 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2726532320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:01.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:01 smithi106 ceph-mon[28081]: pgmap v385: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 127 B/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-04-28T12:51:01.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:01 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/293633588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:01.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:01 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1329039064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:01.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:01 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2726532320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:01.594 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.598+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e0402dd80 con 0x7f1e04104960 2024-04-28T12:51:01.598 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.602+0000 7f1dfaa00640 1 -- 172.21.15.106:0/3585476330 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24928 (secure 0 0 0) 0x7f1df409cbf0 con 0x7f1e04104960 2024-04-28T12:51:01.599 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:01.599 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":93,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:50:58.139109+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.604+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1de80a3c40 msgr2=0x7f1de80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.604+0000 7f1e03200640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1de80a3c40 0x7f1de80a6100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1dec031500 tx=0x7f1dec005a20 comp rx=0 tx=0).stop 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04104960 msgr2=0x7f1e0419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04104960 0x7f1e0419abd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1df400e9c0 tx=0x7f1df400ee80 comp rx=0 tx=0).stop 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 shutdown_connections 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1de80a3c40 0x7f1de80a6100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e04106510 0x7f1e041a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e04105b50 0x7f1e0419b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 --2- 172.21.15.106:0/3585476330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e04104960 0x7f1e0419abd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 >> 172.21.15.106:0/3585476330 conn(0x7f1e040ffeb0 msgr2=0x7f1e04108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:01.602 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 shutdown_connections 2024-04-28T12:51:01.603 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.605+0000 7f1e03200640 1 -- 172.21.15.106:0/3585476330 wait complete. 2024-04-28T12:51:01.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:48:52.916511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:48:53.528808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:48:57.154563+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:51:01.623 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:51:01.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.825+0000 7f5b0f200640 1 -- 172.21.15.106:0/1673807043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 msgr2=0x7f5b101112a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.822 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.825+0000 7f5b0f200640 1 --2- 172.21.15.106:0/1673807043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 0x7f5b101112a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009ca0 tx=0x7f5b0802a8c0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.825+0000 7f5b0f200640 1 -- 172.21.15.106:0/1673807043 shutdown_connections 2024-04-28T12:51:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.825+0000 7f5b0f200640 1 --2- 172.21.15.106:0/1673807043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 0x7f5b101112a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.825+0000 7f5b0f200640 1 --2- 172.21.15.106:0/1673807043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b10109fc0 0x7f5b1010a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.825+0000 7f5b0f200640 1 --2- 172.21.15.106:0/1673807043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b10108e70 0x7f5b10109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.825+0000 7f5b0f200640 1 -- 172.21.15.106:0/1673807043 >> 172.21.15.106:0/1673807043 conn(0x7f5b10077e00 msgr2=0x7f5b1007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.825+0000 7f5b0f200640 1 -- 172.21.15.106:0/1673807043 shutdown_connections 2024-04-28T12:51:01.823 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.825+0000 7f5b0f200640 1 -- 172.21.15.106:0/1673807043 wait complete. 2024-04-28T12:51:01.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.826+0000 7f5b0f200640 1 Processor -- start 2024-04-28T12:51:01.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0f200640 1 -- start start 2024-04-28T12:51:01.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b10108e70 0x7f5b1019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:01.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b10109fc0 0x7f5b1019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:01.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 0x7f5b101a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:01.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b101149f0 con 0x7f5b1010a980 2024-04-28T12:51:01.825 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b10114870 con 0x7f5b10108e70 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5b10114b70 con 0x7f5b10109fc0 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 0x7f5b101a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b10108e70 0x7f5b1019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 0x7f5b101a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36104/0 (socket says 172.21.15.106:36104) 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b0e800640 1 -- 172.21.15.106:0/3665050840 learned_addr learned my addr 172.21.15.106:0/3665050840 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.827+0000 7f5b07e00640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b10109fc0 0x7f5b1019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b0e800640 1 -- 172.21.15.106:0/3665050840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b10109fc0 msgr2=0x7f5b1019f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b0e800640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b10109fc0 0x7f5b1019f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b0e800640 1 -- 172.21.15.106:0/3665050840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b10108e70 msgr2=0x7f5b1019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b0e800640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b10108e70 0x7f5b1019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b0e800640 1 -- 172.21.15.106:0/3665050840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b101a6c00 con 0x7f5b1010a980 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b0e800640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 0x7f5b101a6590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b08033040 tx=0x7f5b080316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:01.826 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b06a00640 1 -- 172.21.15.106:0/3665050840 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08003b70 con 0x7f5b1010a980 2024-04-28T12:51:01.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b06a00640 1 -- 172.21.15.106:0/3665050840 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b08003d10 con 0x7f5b1010a980 2024-04-28T12:51:01.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b06a00640 1 -- 172.21.15.106:0/3665050840 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0802fb40 con 0x7f5b1010a980 2024-04-28T12:51:01.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b101a6e90 con 0x7f5b1010a980 2024-04-28T12:51:01.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.828+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b101a7350 con 0x7f5b1010a980 2024-04-28T12:51:01.827 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.830+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b10110120 con 0x7f5b1010a980 2024-04-28T12:51:01.828 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.832+0000 7f5b06a00640 1 -- 172.21.15.106:0/3665050840 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5b08004610 con 0x7f5b1010a980 2024-04-28T12:51:01.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.834+0000 7f5b06a00640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5af40a3c40 0x7f5af40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:01.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.834+0000 7f5b06a00640 1 -- 172.21.15.106:0/3665050840 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f5b080f1c50 con 0x7f5b1010a980 2024-04-28T12:51:01.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.834+0000 7f5b0de00640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5af40a3c40 0x7f5af40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:01.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.835+0000 7f5b0de00640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5af40a3c40 0x7f5af40a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b000095f0 tx=0x7f5b00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:01.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:01.838+0000 7f5b06a00640 1 -- 172.21.15.106:0/3665050840 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f5b080be1d0 con 0x7f5b1010a980 2024-04-28T12:51:02.037 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.040+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b10063530 con 0x7f5b1010a980 2024-04-28T12:51:02.038 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.041+0000 7f5b06a00640 1 -- 172.21.15.106:0/3665050840 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f5b080bdad0 con 0x7f5b1010a980 2024-04-28T12:51:02.038 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T12:51:02.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5af40a3c40 msgr2=0x7f5af40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5af40a3c40 0x7f5af40a6100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5b000095f0 tx=0x7f5b00005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:02.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 msgr2=0x7f5b101a6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 0x7f5b101a6590 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b08033040 tx=0x7f5b080316f0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 shutdown_connections 2024-04-28T12:51:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f5af40a3c40 0x7f5af40a6100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b1010a980 0x7f5b101a6590 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b10109fc0 0x7f5b1019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 --2- 172.21.15.106:0/3665050840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b10108e70 0x7f5b1019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 >> 172.21.15.106:0/3665050840 conn(0x7f5b10077e00 msgr2=0x7f5b1010d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 shutdown_connections 2024-04-28T12:51:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.044+0000 7f5b0f200640 1 -- 172.21.15.106:0/3665050840 wait complete. 2024-04-28T12:51:02.060 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:51:02.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.270+0000 7ff039000640 1 -- 172.21.15.106:0/1116883011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 msgr2=0x7ff034079770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.270+0000 7ff039000640 1 --2- 172.21.15.106:0/1116883011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 0x7ff034079770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009ca0 tx=0x7ff02c02a8c0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.271+0000 7ff039000640 1 -- 172.21.15.106:0/1116883011 shutdown_connections 2024-04-28T12:51:02.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.271+0000 7ff039000640 1 --2- 172.21.15.106:0/1116883011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 0x7ff034079770 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.271+0000 7ff039000640 1 --2- 172.21.15.106:0/1116883011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff034078530 0x7ff034078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.271+0000 7ff039000640 1 --2- 172.21.15.106:0/1116883011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0340772f0 0x7ff034077710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.271+0000 7ff039000640 1 -- 172.21.15.106:0/1116883011 >> 172.21.15.106:0/1116883011 conn(0x7ff034100120 msgr2=0x7ff034102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:02.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.271+0000 7ff039000640 1 -- 172.21.15.106:0/1116883011 shutdown_connections 2024-04-28T12:51:02.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.271+0000 7ff039000640 1 -- 172.21.15.106:0/1116883011 wait complete. 2024-04-28T12:51:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff039000640 1 Processor -- start 2024-04-28T12:51:02.270 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff039000640 1 -- start start 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff039000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0340772f0 0x7ff03419f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff039000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff034078530 0x7ff03419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff039000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 0x7ff0341a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff039000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff034114b10 con 0x7ff034078ed0 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff039000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff034114990 con 0x7ff0340772f0 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff039000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff034114c90 con 0x7ff034078530 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff032800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 0x7ff0341a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff031e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0340772f0 0x7ff03419f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff032800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 0x7ff0341a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36126/0 (socket says 172.21.15.106:36126) 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.272+0000 7ff032800640 1 -- 172.21.15.106:0/3393810015 learned_addr learned my addr 172.21.15.106:0/3393810015 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.273+0000 7ff029400640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff034078530 0x7ff03419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.273+0000 7ff032800640 1 -- 172.21.15.106:0/3393810015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff034078530 msgr2=0x7ff03419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.273+0000 7ff032800640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff034078530 0x7ff03419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.273+0000 7ff032800640 1 -- 172.21.15.106:0/3393810015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0340772f0 msgr2=0x7ff03419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.273+0000 7ff032800640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0340772f0 0x7ff03419f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.273+0000 7ff032800640 1 -- 172.21.15.106:0/3393810015 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0341a6de0 con 0x7ff034078ed0 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.273+0000 7ff029400640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff034078530 0x7ff03419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.273+0000 7ff031e00640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0340772f0 0x7ff03419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.273+0000 7ff032800640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 0x7ff0341a6770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff02c033040 tx=0x7ff02c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.274+0000 7ff02b400640 1 -- 172.21.15.106:0/3393810015 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c003b70 con 0x7ff034078ed0 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.274+0000 7ff02b400640 1 -- 172.21.15.106:0/3393810015 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff02c003d10 con 0x7ff034078ed0 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.274+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0341a7070 con 0x7ff034078ed0 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.274+0000 7ff02b400640 1 -- 172.21.15.106:0/3393810015 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02c02fb40 con 0x7ff034078ed0 2024-04-28T12:51:02.272 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.274+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0341a75c0 con 0x7ff034078ed0 2024-04-28T12:51:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.275+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff034111500 con 0x7ff034078ed0 2024-04-28T12:51:02.273 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.277+0000 7ff02b400640 1 -- 172.21.15.106:0/3393810015 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff02c004610 con 0x7ff034078ed0 2024-04-28T12:51:02.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.278+0000 7ff02b400640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0180a3c40 0x7ff0180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:02.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.278+0000 7ff02b400640 1 -- 172.21.15.106:0/3393810015 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7ff02c0f1c30 con 0x7ff034078ed0 2024-04-28T12:51:02.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.279+0000 7ff031e00640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0180a3c40 0x7ff0180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:02.275 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.279+0000 7ff031e00640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0180a3c40 0x7ff0180a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff024009fc0 tx=0x7ff024005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:02.279 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.282+0000 7ff02b400640 1 -- 172.21.15.106:0/3393810015 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff02c0be1b0 con 0x7ff034078ed0 2024-04-28T12:51:02.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:02 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3585476330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:02.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:02 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3665050840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:02.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.487+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff034063530 con 0x7ff034078ed0 2024-04-28T12:51:02.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.488+0000 7ff02b400640 1 -- 172.21.15.106:0/3393810015 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7ff02c0bdab0 con 0x7ff034078ed0 2024-04-28T12:51:02.485 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:51:02.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.490+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0180a3c40 msgr2=0x7ff0180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.490+0000 7ff039000640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0180a3c40 0x7ff0180a6100 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff024009fc0 tx=0x7ff024005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.490+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 msgr2=0x7ff0341a6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.491+0000 7ff039000640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 0x7ff0341a6770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff02c033040 tx=0x7ff02c0316f0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.491+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 shutdown_connections 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.491+0000 7ff039000640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff0180a3c40 0x7ff0180a6100 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.491+0000 7ff039000640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff034078ed0 0x7ff0341a6770 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.491+0000 7ff039000640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff034078530 0x7ff03419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.491+0000 7ff039000640 1 --2- 172.21.15.106:0/3393810015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0340772f0 0x7ff03419f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.491+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 >> 172.21.15.106:0/3393810015 conn(0x7ff034100120 msgr2=0x7ff034101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.491+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 shutdown_connections 2024-04-28T12:51:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.491+0000 7ff039000640 1 -- 172.21.15.106:0/3393810015 wait complete. 2024-04-28T12:51:02.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:02 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3585476330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:02.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:02 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3665050840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:02.506 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:51:02.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:02 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3585476330' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:02.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:02 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3665050840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:02.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.714+0000 7fc31f200640 1 -- 172.21.15.106:0/2133450725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 msgr2=0x7fc3201091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.714+0000 7fc31f200640 1 --2- 172.21.15.106:0/2133450725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 0x7fc3201091b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc310009860 tx=0x7fc31002a340 comp rx=0 tx=0).stop 2024-04-28T12:51:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.714+0000 7fc31f200640 1 -- 172.21.15.106:0/2133450725 shutdown_connections 2024-04-28T12:51:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.714+0000 7fc31f200640 1 --2- 172.21.15.106:0/2133450725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32010a940 0x7fc3201111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.714+0000 7fc31f200640 1 --2- 172.21.15.106:0/2133450725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc320109f80 0x7fc32010a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.714+0000 7fc31f200640 1 --2- 172.21.15.106:0/2133450725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 0x7fc3201091b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.714+0000 7fc31f200640 1 -- 172.21.15.106:0/2133450725 >> 172.21.15.106:0/2133450725 conn(0x7fc320077d30 msgr2=0x7fc32007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.715+0000 7fc31f200640 1 -- 172.21.15.106:0/2133450725 shutdown_connections 2024-04-28T12:51:02.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.715+0000 7fc31f200640 1 -- 172.21.15.106:0/2133450725 wait complete. 2024-04-28T12:51:02.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.716+0000 7fc31f200640 1 Processor -- start 2024-04-28T12:51:02.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.716+0000 7fc31f200640 1 -- start start 2024-04-28T12:51:02.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.716+0000 7fc31f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 0x7fc32019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:02.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.716+0000 7fc31f200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc320109f80 0x7fc32019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:02.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.716+0000 7fc31f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32010a940 0x7fc3201a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:02.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.716+0000 7fc31f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc320114890 con 0x7fc32010a940 2024-04-28T12:51:02.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.716+0000 7fc31f200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc320114710 con 0x7fc320109f80 2024-04-28T12:51:02.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.716+0000 7fc31f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc320114a10 con 0x7fc320108d90 2024-04-28T12:51:02.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31de00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 0x7fc32019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31d400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc320109f80 0x7fc32019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31d400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc320109f80 0x7fc32019f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:32876/0 (socket says 172.21.15.106:32876) 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31de00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 0x7fc32019f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:47754/0 (socket says 172.21.15.106:47754) 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31d400640 1 -- 172.21.15.106:0/1873416897 learned_addr learned my addr 172.21.15.106:0/1873416897 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31e800640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32010a940 0x7fc3201a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31de00640 1 -- 172.21.15.106:0/1873416897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc320109f80 msgr2=0x7fc32019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31de00640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc320109f80 0x7fc32019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31de00640 1 -- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32010a940 msgr2=0x7fc3201a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31de00640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32010a940 0x7fc3201a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31de00640 1 -- 172.21.15.106:0/1873416897 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3201a6bf0 con 0x7fc320108d90 2024-04-28T12:51:02.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.717+0000 7fc31de00640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 0x7fc32019f020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc31002a860 tx=0x7fc310032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.718+0000 7fc316a00640 1 -- 172.21.15.106:0/1873416897 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310040020 con 0x7fc320108d90 2024-04-28T12:51:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.718+0000 7fc316a00640 1 -- 172.21.15.106:0/1873416897 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc310030d50 con 0x7fc320108d90 2024-04-28T12:51:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.718+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3201a6e80 con 0x7fc320108d90 2024-04-28T12:51:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.718+0000 7fc316a00640 1 -- 172.21.15.106:0/1873416897 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310037040 con 0x7fc320108d90 2024-04-28T12:51:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.718+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3201a7280 con 0x7fc320108d90 2024-04-28T12:51:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.719+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc320110220 con 0x7fc320108d90 2024-04-28T12:51:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.721+0000 7fc316a00640 1 -- 172.21.15.106:0/1873416897 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc310043020 con 0x7fc320108d90 2024-04-28T12:51:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.722+0000 7fc316a00640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc3040a3c40 0x7fc3040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.723+0000 7fc316a00640 1 -- 172.21.15.106:0/1873416897 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fc3100f0c00 con 0x7fc320108d90 2024-04-28T12:51:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.723+0000 7fc31d400640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc3040a3c40 0x7fc3040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:02.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.723+0000 7fc31d400640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc3040a3c40 0x7fc3040a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc3201a04e0 tx=0x7fc308005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.725+0000 7fc316a00640 1 -- 172.21.15.106:0/1873416897 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fc31004a020 con 0x7fc320108d90 2024-04-28T12:51:02.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.923+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc32002dd20 con 0x7fc320108d90 2024-04-28T12:51:02.920 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.924+0000 7fc316a00640 1 -- 172.21.15.106:0/1873416897 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fc3100bd180 con 0x7fc320108d90 2024-04-28T12:51:02.920 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-28T12:51:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc3040a3c40 msgr2=0x7fc3040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc3040a3c40 0x7fc3040a6100 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc3201a04e0 tx=0x7fc308005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 msgr2=0x7fc32019f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 0x7fc32019f020 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc31002a860 tx=0x7fc310032d50 comp rx=0 tx=0).stop 2024-04-28T12:51:02.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 shutdown_connections 2024-04-28T12:51:02.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fc3040a3c40 0x7fc3040a6100 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc32010a940 0x7fc3201a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc320109f80 0x7fc32019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 --2- 172.21.15.106:0/1873416897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc320108d90 0x7fc32019f020 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:02.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.926+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 >> 172.21.15.106:0/1873416897 conn(0x7fc320077d30 msgr2=0x7fc32010d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:02.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.927+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 shutdown_connections 2024-04-28T12:51:02.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:02.927+0000 7fc31f200640 1 -- 172.21.15.106:0/1873416897 wait complete. 2024-04-28T12:51:02.942 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:51:03.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.143+0000 7f434ca00640 1 -- 172.21.15.106:0/2600226365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348103a30 msgr2=0x7f4348103eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.143+0000 7f434ca00640 1 --2- 172.21.15.106:0/2600226365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348103a30 0x7f4348103eb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4338009900 tx=0x7f433802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.143+0000 7f434ca00640 1 -- 172.21.15.106:0/2600226365 shutdown_connections 2024-04-28T12:51:03.140 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.143+0000 7f434ca00640 1 --2- 172.21.15.106:0/2600226365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348103a30 0x7f4348103eb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.143+0000 7f434ca00640 1 --2- 172.21.15.106:0/2600226365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43481043f0 0x7f434810ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.143+0000 7f434ca00640 1 --2- 172.21.15.106:0/2600226365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4348102840 0x7f4348102c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.143+0000 7f434ca00640 1 -- 172.21.15.106:0/2600226365 >> 172.21.15.106:0/2600226365 conn(0x7f43480fddd0 msgr2=0x7f43481001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:03.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.143+0000 7f434ca00640 1 -- 172.21.15.106:0/2600226365 shutdown_connections 2024-04-28T12:51:03.141 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.143+0000 7f434ca00640 1 -- 172.21.15.106:0/2600226365 wait complete. 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.144+0000 7f434ca00640 1 Processor -- start 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.144+0000 7f434ca00640 1 -- start start 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.144+0000 7f434ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348102840 0x7f434819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.144+0000 7f434ca00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4348103a30 0x7f434819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.144+0000 7f434ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43481043f0 0x7f43481a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.144+0000 7f434ca00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f434810e370 con 0x7f4348102840 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f434ca00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f434810e1f0 con 0x7f4348103a30 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f434ca00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f434810e4f0 con 0x7f43481043f0 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4346800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348102840 0x7f434819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:03.143 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4346800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348102840 0x7f434819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36166/0 (socket says 172.21.15.106:36166) 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4346800640 1 -- 172.21.15.106:0/2183148658 learned_addr learned my addr 172.21.15.106:0/2183148658 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4345e00640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4348103a30 0x7f434819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4347200640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43481043f0 0x7f43481a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4346800640 1 -- 172.21.15.106:0/2183148658 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43481043f0 msgr2=0x7f43481a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4346800640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43481043f0 0x7f43481a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4346800640 1 -- 172.21.15.106:0/2183148658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4348103a30 msgr2=0x7f434819b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4346800640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4348103a30 0x7f434819b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4346800640 1 -- 172.21.15.106:0/2183148658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43481a28c0 con 0x7f4348102840 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.145+0000 7f4346800640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348102840 0x7f434819ac80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f433400d950 tx=0x7f433400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.146+0000 7f433f400640 1 -- 172.21.15.106:0/2183148658 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334017070 con 0x7f4348102840 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.146+0000 7f433f400640 1 -- 172.21.15.106:0/2183148658 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4334009c30 con 0x7f4348102840 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.146+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43481a2bb0 con 0x7f4348102840 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.146+0000 7f433f400640 1 -- 172.21.15.106:0/2183148658 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433400f620 con 0x7f4348102840 2024-04-28T12:51:03.144 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.146+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43481a3080 con 0x7f4348102840 2024-04-28T12:51:03.145 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.147+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43481a3250 con 0x7f4348102840 2024-04-28T12:51:03.146 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.149+0000 7f433f400640 1 -- 172.21.15.106:0/2183148658 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f433401a070 con 0x7f4348102840 2024-04-28T12:51:03.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.151+0000 7f433f400640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f432c0a3c40 0x7f432c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:03.147 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.151+0000 7f433f400640 1 -- 172.21.15.106:0/2183148658 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f43340d0370 con 0x7f4348102840 2024-04-28T12:51:03.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.151+0000 7f4345e00640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f432c0a3c40 0x7f432c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:03.148 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.152+0000 7f4345e00640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f432c0a3c40 0x7f432c0a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4338000c00 tx=0x7f43380314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:03.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.155+0000 7f433f400640 1 -- 172.21.15.106:0/2183148658 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4334009de0 con 0x7f4348102840 2024-04-28T12:51:03.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.356+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f43481a3540 con 0x7f4348102840 2024-04-28T12:51:03.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.357+0000 7f433f400640 1 -- 172.21.15.106:0/2183148658 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f433409c190 con 0x7f4348102840 2024-04-28T12:51:03.354 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:51:03.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.359+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f432c0a3c40 msgr2=0x7f432c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.359+0000 7f434ca00640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f432c0a3c40 0x7f432c0a6100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4338000c00 tx=0x7f43380314a0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348102840 msgr2=0x7f434819ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348102840 0x7f434819ac80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f433400d950 tx=0x7f433400de10 comp rx=0 tx=0).stop 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 shutdown_connections 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f432c0a3c40 0x7f432c0a6100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43481043f0 0x7f43481a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4348103a30 0x7f434819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 --2- 172.21.15.106:0/2183148658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4348102840 0x7f434819ac80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 >> 172.21.15.106:0/2183148658 conn(0x7f43480fddd0 msgr2=0x7f4348109200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 shutdown_connections 2024-04-28T12:51:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.360+0000 7f434ca00640 1 -- 172.21.15.106:0/2183148658 wait complete. 2024-04-28T12:51:03.376 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:51:03.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:03 smithi171 ceph-mon[63127]: pgmap v386: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-28T12:51:03.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:03 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3393810015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:03.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:03 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1873416897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:03.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:03 smithi129 ceph-mon[62418]: pgmap v386: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-28T12:51:03.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:03 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3393810015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:03.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:03 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1873416897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:03.561 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.564+0000 7f4665000640 1 -- 172.21.15.106:0/570520976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4660104b90 msgr2=0x7f4660104fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.564+0000 7f4665000640 1 --2- 172.21.15.106:0/570520976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4660104b90 0x7f4660104fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4650009860 tx=0x7f465002a340 comp rx=0 tx=0).stop 2024-04-28T12:51:03.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.565+0000 7f4665000640 1 -- 172.21.15.106:0/570520976 shutdown_connections 2024-04-28T12:51:03.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.565+0000 7f4665000640 1 --2- 172.21.15.106:0/570520976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4660106740 0x7f466010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.565+0000 7f4665000640 1 --2- 172.21.15.106:0/570520976 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4660105d80 0x7f4660106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.565+0000 7f4665000640 1 --2- 172.21.15.106:0/570520976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4660104b90 0x7f4660104fb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.565+0000 7f4665000640 1 -- 172.21.15.106:0/570520976 >> 172.21.15.106:0/570520976 conn(0x7f4660100120 msgr2=0x7f4660102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:03.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.565+0000 7f4665000640 1 -- 172.21.15.106:0/570520976 shutdown_connections 2024-04-28T12:51:03.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.565+0000 7f4665000640 1 -- 172.21.15.106:0/570520976 wait complete. 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f4665000640 1 Processor -- start 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f4665000640 1 -- start start 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f4665000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4660104b90 0x7f466019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f4665000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4660105d80 0x7f466019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f4665000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4660106740 0x7f46601a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f4665000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4660110690 con 0x7f4660104b90 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f4665000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4660110510 con 0x7f4660105d80 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f4665000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4660110810 con 0x7f4660106740 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f465de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4660104b90 0x7f466019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:03.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f465de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4660104b90 0x7f466019adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36182/0 (socket says 172.21.15.106:36182) 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f465de00640 1 -- 172.21.15.106:0/3388399080 learned_addr learned my addr 172.21.15.106:0/3388399080 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f465d400640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4660105d80 0x7f466019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.566+0000 7f465e800640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4660106740 0x7f46601a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f465de00640 1 -- 172.21.15.106:0/3388399080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4660106740 msgr2=0x7f46601a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f465de00640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4660106740 0x7f46601a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f465de00640 1 -- 172.21.15.106:0/3388399080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4660105d80 msgr2=0x7f466019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f465de00640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4660105d80 0x7f466019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f465de00640 1 -- 172.21.15.106:0/3388399080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46601a29e0 con 0x7f4660104b90 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f465de00640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4660104b90 0x7f466019adc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46500057c0 tx=0x7f4650032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f4656a00640 1 -- 172.21.15.106:0/3388399080 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650037040 con 0x7f4660104b90 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f4656a00640 1 -- 172.21.15.106:0/3388399080 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4650030bd0 con 0x7f4660104b90 2024-04-28T12:51:03.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46601a2c70 con 0x7f4660104b90 2024-04-28T12:51:03.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f4656a00640 1 -- 172.21.15.106:0/3388399080 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650005280 con 0x7f4660104b90 2024-04-28T12:51:03.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.567+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46601a3100 con 0x7f4660104b90 2024-04-28T12:51:03.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.568+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466010bf80 con 0x7f4660104b90 2024-04-28T12:51:03.567 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.570+0000 7f4656a00640 1 -- 172.21.15.106:0/3388399080 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4650043020 con 0x7f4660104b90 2024-04-28T12:51:03.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.572+0000 7f4656a00640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f46440a3c40 0x7f46440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:03.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.572+0000 7f4656a00640 1 -- 172.21.15.106:0/3388399080 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f46500efd70 con 0x7f4660104b90 2024-04-28T12:51:03.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.572+0000 7f465d400640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f46440a3c40 0x7f46440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:03.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.573+0000 7f465d400640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f46440a3c40 0x7f46440a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f466019c2d0 tx=0x7f464c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:03.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.576+0000 7f4656a00640 1 -- 172.21.15.106:0/3388399080 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f46500bc330 con 0x7f4660104b90 2024-04-28T12:51:03.574 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:03 smithi106 ceph-mon[28081]: pgmap v386: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s; 14 B/s, 0 objects/s recovering 2024-04-28T12:51:03.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:03 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3393810015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:03.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:03 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1873416897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:03.799 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.802+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f466002dd20 con 0x7f4660104b90 2024-04-28T12:51:03.800 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.803+0000 7f4656a00640 1 -- 172.21.15.106:0/3388399080 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11953 (secure 0 0 0) 0x7f46500bbc30 con 0x7f4660104b90 2024-04-28T12:51:03.800 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:03.800 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:49:30.023676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24548,"mds_3":24538,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24548,24520,24578,24538,24566]},"id":1}]} 2024-04-28T12:51:03.800 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f46440a3c40 msgr2=0x7f46440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f46440a3c40 0x7f46440a6100 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f466019c2d0 tx=0x7f464c005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4660104b90 msgr2=0x7f466019adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4660104b90 0x7f466019adc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f46500057c0 tx=0x7f4650032da0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 shutdown_connections 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f46440a3c40 0x7f46440a6100 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4660106740 0x7f46601a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4660105d80 0x7f466019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 --2- 172.21.15.106:0/3388399080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4660104b90 0x7f466019adc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:03.803 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 >> 172.21.15.106:0/3388399080 conn(0x7f4660100120 msgr2=0x7f4660109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 shutdown_connections 2024-04-28T12:51:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:03.806+0000 7f4665000640 1 -- 172.21.15.106:0/3388399080 wait complete. 2024-04-28T12:51:03.821 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:51:04.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.016+0000 7fe1aae00640 1 -- 172.21.15.106:0/1137336809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a410ab40 msgr2=0x7fe1a41113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.016+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1137336809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a410ab40 0x7fe1a41113d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009ca0 tx=0x7fe19c02a8c0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.016+0000 7fe1aae00640 1 -- 172.21.15.106:0/1137336809 shutdown_connections 2024-04-28T12:51:04.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.016+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1137336809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a410ab40 0x7fe1a41113d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.013 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.016+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1137336809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a410a180 0x7fe1a410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.016+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1137336809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a4108f90 0x7fe1a41093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.016+0000 7fe1aae00640 1 -- 172.21.15.106:0/1137336809 >> 172.21.15.106:0/1137336809 conn(0x7fe1a4077f40 msgr2=0x7fe1a407a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.016+0000 7fe1aae00640 1 -- 172.21.15.106:0/1137336809 shutdown_connections 2024-04-28T12:51:04.014 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.016+0000 7fe1aae00640 1 -- 172.21.15.106:0/1137336809 wait complete. 2024-04-28T12:51:04.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.017+0000 7fe1aae00640 1 Processor -- start 2024-04-28T12:51:04.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.017+0000 7fe1aae00640 1 -- start start 2024-04-28T12:51:04.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.017+0000 7fe1aae00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a4108f90 0x7fe1a419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.017+0000 7fe1aae00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a410a180 0x7fe1a419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1aae00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a410ab40 0x7fe1a41a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1aae00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4114ad0 con 0x7fe1a4108f90 2024-04-28T12:51:04.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1aae00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4114950 con 0x7fe1a410a180 2024-04-28T12:51:04.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1aae00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4114c50 con 0x7fe1a410ab40 2024-04-28T12:51:04.016 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a4108f90 0x7fe1a419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a4108f90 0x7fe1a419f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36202/0 (socket says 172.21.15.106:36202) 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a3e00640 1 -- 172.21.15.106:0/1448770066 learned_addr learned my addr 172.21.15.106:0/1448770066 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe19be00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a410a180 0x7fe1a419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a8e00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a410ab40 0x7fe1a41a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a3e00640 1 -- 172.21.15.106:0/1448770066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a410ab40 msgr2=0x7fe1a41a66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a3e00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a410ab40 0x7fe1a41a66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a3e00640 1 -- 172.21.15.106:0/1448770066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a410a180 msgr2=0x7fe1a419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a3e00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a410a180 0x7fe1a419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a3e00640 1 -- 172.21.15.106:0/1448770066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a41a6d40 con 0x7fe1a4108f90 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a8e00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a410ab40 0x7fe1a41a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.018+0000 7fe1a3e00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a4108f90 0x7fe1a419f120 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe19400e9c0 tx=0x7fe19400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.019+0000 7fe1a1600640 1 -- 172.21.15.106:0/1448770066 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19400cd80 con 0x7fe1a4108f90 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.019+0000 7fe1a1600640 1 -- 172.21.15.106:0/1448770066 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1940035f0 con 0x7fe1a4108f90 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.019+0000 7fe1a1600640 1 -- 172.21.15.106:0/1448770066 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194010690 con 0x7fe1a4108f90 2024-04-28T12:51:04.017 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.019+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a41a6fd0 con 0x7fe1a4108f90 2024-04-28T12:51:04.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.019+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a41a74e0 con 0x7fe1a4108f90 2024-04-28T12:51:04.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.020+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1a41102b0 con 0x7fe1a4108f90 2024-04-28T12:51:04.018 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.022+0000 7fe1a1600640 1 -- 172.21.15.106:0/1448770066 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe1940037a0 con 0x7fe1a4108f90 2024-04-28T12:51:04.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.023+0000 7fe1a1600640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1880a3cc0 0x7fe1880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.023+0000 7fe1a1600640 1 -- 172.21.15.106:0/1448770066 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fe1940d0d50 con 0x7fe1a4108f90 2024-04-28T12:51:04.020 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.024+0000 7fe19be00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1880a3cc0 0x7fe1880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.024+0000 7fe19be00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1880a3cc0 0x7fe1880a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe1a41a0630 tx=0x7fe18c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.028+0000 7fe1a1600640 1 -- 172.21.15.106:0/1448770066 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe19409d1c0 con 0x7fe1a4108f90 2024-04-28T12:51:04.231 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.234+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1a40635d0 con 0x7fe1a4108f90 2024-04-28T12:51:04.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.238+0000 7fe1a1600640 1 -- 172.21.15.106:0/1448770066 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24928 (secure 0 0 0) 0x7fe19409cac0 con 0x7fe1a4108f90 2024-04-28T12:51:04.236 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:04.236 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":93,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:50:58.139109+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:51:04.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1880a3cc0 msgr2=0x7fe1880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1880a3cc0 0x7fe1880a6180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe1a41a0630 tx=0x7fe18c005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:04.238 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a4108f90 msgr2=0x7fe1a419f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a4108f90 0x7fe1a419f120 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe19400e9c0 tx=0x7fe19400ee80 comp rx=0 tx=0).stop 2024-04-28T12:51:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 shutdown_connections 2024-04-28T12:51:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe1880a3cc0 0x7fe1880a6180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1a410ab40 0x7fe1a41a66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1a410a180 0x7fe1a419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 --2- 172.21.15.106:0/1448770066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1a4108f90 0x7fe1a419f120 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.241+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 >> 172.21.15.106:0/1448770066 conn(0x7fe1a4077f40 msgr2=0x7fe1a410d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.242+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 shutdown_connections 2024-04-28T12:51:04.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.242+0000 7fe1aae00640 1 -- 172.21.15.106:0/1448770066 wait complete. 2024-04-28T12:51:04.257 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:51:04.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.446+0000 7fe910e00640 1 -- 172.21.15.106:0/832597733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe90c106510 msgr2=0x7fe90c10cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.446+0000 7fe910e00640 1 --2- 172.21.15.106:0/832597733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe90c106510 0x7fe90c10cda0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe904009ca0 tx=0x7fe90402a8c0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.446+0000 7fe910e00640 1 -- 172.21.15.106:0/832597733 shutdown_connections 2024-04-28T12:51:04.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.446+0000 7fe910e00640 1 --2- 172.21.15.106:0/832597733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe90c106510 0x7fe90c10cda0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.446+0000 7fe910e00640 1 --2- 172.21.15.106:0/832597733 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe90c105b50 0x7fe90c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.446+0000 7fe910e00640 1 --2- 172.21.15.106:0/832597733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe90c104960 0x7fe90c104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.446+0000 7fe910e00640 1 -- 172.21.15.106:0/832597733 >> 172.21.15.106:0/832597733 conn(0x7fe90c0ffeb0 msgr2=0x7fe90c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.446+0000 7fe910e00640 1 -- 172.21.15.106:0/832597733 shutdown_connections 2024-04-28T12:51:04.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.446+0000 7fe910e00640 1 -- 172.21.15.106:0/832597733 wait complete. 2024-04-28T12:51:04.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.447+0000 7fe910e00640 1 Processor -- start 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.447+0000 7fe910e00640 1 -- start start 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.447+0000 7fe910e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe90c104960 0x7fe90c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.447+0000 7fe910e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe90c105b50 0x7fe90c19b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.447+0000 7fe910e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe90c106510 0x7fe90c1a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.447+0000 7fe910e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c110490 con 0x7fe90c106510 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.447+0000 7fe910e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c110310 con 0x7fe90c104960 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.447+0000 7fe910e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c110610 con 0x7fe90c105b50 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe90c106510 0x7fe90c1a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe90c104960 0x7fe90c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe90c106510 0x7fe90c1a20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36208/0 (socket says 172.21.15.106:36208) 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90b400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe90c104960 0x7fe90c19ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:32948/0 (socket says 172.21.15.106:32948) 2024-04-28T12:51:04.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90be00640 1 -- 172.21.15.106:0/3846376445 learned_addr learned my addr 172.21.15.106:0/3846376445 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe90c105b50 0x7fe90c19b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90b400640 1 -- 172.21.15.106:0/3846376445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe90c105b50 msgr2=0x7fe90c19b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90b400640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe90c105b50 0x7fe90c19b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90b400640 1 -- 172.21.15.106:0/3846376445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe90c106510 msgr2=0x7fe90c1a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90b400640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe90c106510 0x7fe90c1a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90b400640 1 -- 172.21.15.106:0/3846376445 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c1a2730 con 0x7fe90c104960 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.448+0000 7fe90b400640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe90c104960 0x7fe90c19ab10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00e9c0 tx=0x7fe8fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.449+0000 7fe903e00640 1 -- 172.21.15.106:0/3846376445 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00ccc0 con 0x7fe90c104960 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.449+0000 7fe903e00640 1 -- 172.21.15.106:0/3846376445 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8fc0035f0 con 0x7fe90c104960 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.449+0000 7fe903e00640 1 -- 172.21.15.106:0/3846376445 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00ce60 con 0x7fe90c104960 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.449+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c1a2a20 con 0x7fe90c104960 2024-04-28T12:51:04.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.449+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c1a2f20 con 0x7fe90c104960 2024-04-28T12:51:04.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.450+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90c10bc80 con 0x7fe90c104960 2024-04-28T12:51:04.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.452+0000 7fe903e00640 1 -- 172.21.15.106:0/3846376445 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe8fc004000 con 0x7fe90c104960 2024-04-28T12:51:04.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.453+0000 7fe903e00640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe8f00a3cc0 0x7fe8f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.454+0000 7fe903e00640 1 -- 172.21.15.106:0/3846376445 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fe8fc0d0d00 con 0x7fe90c104960 2024-04-28T12:51:04.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.454+0000 7fe90aa00640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe8f00a3cc0 0x7fe8f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.454+0000 7fe90aa00640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe8f00a3cc0 0x7fe8f00a6180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe90c19c020 tx=0x7fe8f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.458+0000 7fe903e00640 1 -- 172.21.15.106:0/3846376445 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe8fc09d170 con 0x7fe90c104960 2024-04-28T12:51:04.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:04 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2183148658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:04.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:04 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3388399080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:04.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:04 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2183148658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:04.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:04 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3388399080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:04.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:04 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2183148658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:04.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:04 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3388399080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:04.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.681+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe90c02dad0 con 0x7fe90c104960 2024-04-28T12:51:04.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.682+0000 7fe903e00640 1 -- 172.21.15.106:0/3846376445 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11953 (secure 0 0 0) 0x7fe8fc09ca70 con 0x7fe90c104960 2024-04-28T12:51:04.679 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:04.679 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:49:30.023676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24548,"mds_3":24538,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24548,24520,24578,24538,24566]},"id":1}]} 2024-04-28T12:51:04.679 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-28T12:51:04.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe8f00a3cc0 msgr2=0x7fe8f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe8f00a3cc0 0x7fe8f00a6180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe90c19c020 tx=0x7fe8f8005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:04.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe90c104960 msgr2=0x7fe90c19ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe90c104960 0x7fe90c19ab10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00e9c0 tx=0x7fe8fc00ee80 comp rx=0 tx=0).stop 2024-04-28T12:51:04.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 shutdown_connections 2024-04-28T12:51:04.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe8f00a3cc0 0x7fe8f00a6180 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe90c106510 0x7fe90c1a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe90c105b50 0x7fe90c19b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 --2- 172.21.15.106:0/3846376445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe90c104960 0x7fe90c19ab10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.684+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 >> 172.21.15.106:0/3846376445 conn(0x7fe90c0ffeb0 msgr2=0x7fe90c108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.685+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 shutdown_connections 2024-04-28T12:51:04.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.685+0000 7fe910e00640 1 -- 172.21.15.106:0/3846376445 wait complete. 2024-04-28T12:51:04.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-28T12:48:54.598061+0000', 'modified': '2024-04-28T12:49:30.023676+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24578, 'mds_1': 24566, 'mds_2': 24548, 'mds_3': 24538, 'mds_4': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'j', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6835/3035871722', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 3035871722}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 3035871722}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'd', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/4131098763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 4131098763}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 4131098763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6835/1242652051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 1242652051}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 1242652051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6835/4221893526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 4221893526}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 4221893526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6837/1065884016', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 1065884016}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 1065884016}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.171:6837/813345311', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 813345311}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 813345311}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.171:6839/2406303763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 2406303763}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 2406303763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'g', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.106:6839/3863798001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 3863798001}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 3863798001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'a', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.106:6841/92504617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6840', 'nonce': 92504617}, {'type': 'v1', 'addr': '172.21.15.106:6841', 'nonce': 92504617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.129:6839/4208792665', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 4208792665}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 4208792665}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24578, 'qdb_cluster': [24548, 24520, 24578, 24538, 24566]} 2024-04-28T12:51:04.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-28T12:51:04.700 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-28T12:51:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.902+0000 7f17dc000640 1 -- 172.21.15.106:0/1045694140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 msgr2=0x7f17d4075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.902+0000 7f17dc000640 1 --2- 172.21.15.106:0/1045694140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 0x7f17d4075810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009900 tx=0x7f17c002a3f0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.903+0000 7f17dc000640 1 -- 172.21.15.106:0/1045694140 shutdown_connections 2024-04-28T12:51:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.903+0000 7f17dc000640 1 --2- 172.21.15.106:0/1045694140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 0x7f17d4075810 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.903+0000 7f17dc000640 1 --2- 172.21.15.106:0/1045694140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17d410a840 0x7f17d410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.903+0000 7f17dc000640 1 --2- 172.21.15.106:0/1045694140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17d4076970 0x7f17d4074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.903+0000 7f17dc000640 1 -- 172.21.15.106:0/1045694140 >> 172.21.15.106:0/1045694140 conn(0x7f17d40fdf10 msgr2=0x7f17d4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.903+0000 7f17dc000640 1 -- 172.21.15.106:0/1045694140 shutdown_connections 2024-04-28T12:51:04.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.903+0000 7f17dc000640 1 -- 172.21.15.106:0/1045694140 wait complete. 2024-04-28T12:51:04.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17dc000640 1 Processor -- start 2024-04-28T12:51:04.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17dc000640 1 -- start start 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17dc000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 0x7f17d419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17dc000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17d4076970 0x7f17d419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17dc000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17d410a840 0x7f17d41a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17dc000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41106e0 con 0x7f17d4075390 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17dc000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4110560 con 0x7f17d4076970 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17dc000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4110860 con 0x7f17d410a840 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17d9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 0x7f17d419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17d9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 0x7f17d419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36236/0 (socket says 172.21.15.106:36236) 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.904+0000 7f17d9600640 1 -- 172.21.15.106:0/3756569512 learned_addr learned my addr 172.21.15.106:0/3756569512 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.905+0000 7f17da000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17d410a840 0x7f17d41a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.905+0000 7f17d8c00640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17d4076970 0x7f17d419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.905+0000 7f17d9600640 1 -- 172.21.15.106:0/3756569512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17d410a840 msgr2=0x7f17d41a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.905+0000 7f17d9600640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17d410a840 0x7f17d41a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.905+0000 7f17d9600640 1 -- 172.21.15.106:0/3756569512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17d4076970 msgr2=0x7f17d419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.905+0000 7f17d9600640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17d4076970 0x7f17d419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.905+0000 7f17d9600640 1 -- 172.21.15.106:0/3756569512 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d41a29e0 con 0x7f17d4075390 2024-04-28T12:51:04.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.905+0000 7f17d9600640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 0x7f17d419ae10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009d50 tx=0x7f17c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.905+0000 7f17d2000640 1 -- 172.21.15.106:0/3756569512 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4003b70 con 0x7f17d4075390 2024-04-28T12:51:04.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.906+0000 7f17dc000640 1 -- 172.21.15.106:0/3756569512 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d41a2cd0 con 0x7f17d4075390 2024-04-28T12:51:04.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.906+0000 7f17d2000640 1 -- 172.21.15.106:0/3756569512 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c4003d10 con 0x7f17d4075390 2024-04-28T12:51:04.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.906+0000 7f17d2000640 1 -- 172.21.15.106:0/3756569512 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4004d40 con 0x7f17d4075390 2024-04-28T12:51:04.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.906+0000 7f17dc000640 1 -- 172.21.15.106:0/3756569512 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41a3190 con 0x7f17d4075390 2024-04-28T12:51:04.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17dedf2740 1 Processor -- start 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17dedf2740 1 -- start start 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17d2000640 1 -- 172.21.15.106:0/3756569512 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f17c4004000 con 0x7f17d4075390 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fbd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17dedf2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1f1c600 con 0x55e7e1fbdc50 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17dedf2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1eb9100 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17dedf2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1f185c0 con 0x55e7e1fb80f0 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.909+0000 7f17d8c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17d8c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36248/0 (socket says 172.21.15.106:36248) 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17d8c00640 1 -- 172.21.15.106:0/4290930706 learned_addr learned my addr 172.21.15.106:0/4290930706 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:04.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17d9600640 1 --2- 172.21.15.106:0/4290930706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fbd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17da000640 1 --2- 172.21.15.106:0/4290930706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17d8c00640 1 -- 172.21.15.106:0/4290930706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb80f0 msgr2=0x55e7e1fb8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:04.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17d8c00640 1 --2- 172.21.15.106:0/4290930706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17d8c00640 1 -- 172.21.15.106:0/4290930706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb8ae0 msgr2=0x55e7e1fbd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:04.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17d8c00640 1 --2- 172.21.15.106:0/4290930706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fbd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17d8c00640 1 -- 172.21.15.106:0/4290930706 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e1c2e2d0 con 0x55e7e1fbdc50 2024-04-28T12:51:04.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.910+0000 7f17d8c00640 1 --2- 172.21.15.106:0/4290930706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc00e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f17c0006010 tx=0x7f17c0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2e6d52655fd3f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.911+0000 7f17ca600640 1 -- 172.21.15.106:0/4290930706 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0035030 con 0x55e7e1fbdc50 2024-04-28T12:51:04.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.911+0000 7f17ca600640 1 -- 172.21.15.106:0/4290930706 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c0040070 con 0x55e7e1fbdc50 2024-04-28T12:51:04.912 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.911+0000 7f17ca600640 1 -- 172.21.15.106:0/4290930706 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0031890 con 0x55e7e1fbdc50 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.911+0000 7f17d2000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17b80a3c40 0x7f17b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 -- 172.21.15.106:0/4290930706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 msgr2=0x55e7e1fc00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 --2- 172.21.15.106:0/4290930706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc00e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f17c0006010 tx=0x7f17c0004040 comp rx=0 tx=0).stop 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17d2000640 1 -- 172.21.15.106:0/3756569512 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f17c40d0f20 con 0x7f17d4075390 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17da000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17b80a3c40 0x7f17b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 -- 172.21.15.106:0/4290930706 shutdown_connections 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 --2- 172.21.15.106:0/4290930706 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc00e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 --2- 172.21.15.106:0/4290930706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fbd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 --2- 172.21.15.106:0/4290930706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 -- 172.21.15.106:0/4290930706 >> 172.21.15.106:0/4290930706 conn(0x55e7e1f4cd10 msgr2=0x55e7e1fb3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 -- 172.21.15.106:0/4290930706 shutdown_connections 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 -- 172.21.15.106:0/4290930706 wait complete. 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17da000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17b80a3c40 0x7f17b80a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e7e1fb9340 tx=0x7f17cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.912+0000 7f17dedf2740 1 -- start start 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.913+0000 7f17dedf2740 1 Processor -- start 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.913+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.913+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.913+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.913+0000 7f17dedf2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1fc0ba0 con 0x55e7e1fbdc50 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.913+0000 7f17dedf2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1ea3f70 con 0x55e7e1fb80f0 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.913+0000 7f17dedf2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1fc0d20 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.913+0000 7f17dedf2740 10 client.0 _ll_register_callbacks cb 0x55e7e1f12480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.913+0000 7f17da000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17d8c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.914 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17da000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36260/0 (socket says 172.21.15.106:36260) 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17d9600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17da000640 1 -- 172.21.15.106:0/3390552024 learned_addr learned my addr 172.21.15.106:0/3390552024 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17d8c00640 1 -- 172.21.15.106:0/3390552024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb80f0 msgr2=0x55e7e1fc24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17d8c00640 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17d8c00640 1 -- 172.21.15.106:0/3390552024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 msgr2=0x55e7e1fc9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17d8c00640 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17d8c00640 1 -- 172.21.15.106:0/3390552024 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e1cf86b0 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17d8c00640 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc2a20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005ab0 tx=0x7f17c00044e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29fc70b6d0a6a69e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17bea00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.914+0000 7f17bea00640 1 -- 172.21.15.106:0/3390552024 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0053060 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.915+0000 7f17dedf2740 1 -- 172.21.15.106:0/3390552024 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c0040070 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.915+0000 7f17bea00640 1 -- 172.21.15.106:0/3390552024 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f17c0003a00 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.915+0000 7f17bea00640 1 -- 172.21.15.106:0/3390552024 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17c005a050 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.915+0000 7f17dedf2740 10 client.25156 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.915+0000 7f17dedf2740 1 -- 172.21.15.106:0/3390552024 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7e1fc0ba0 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.915+0000 7f17bea00640 1 -- 172.21.15.106:0/3390552024 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c0040070 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.915+0000 7f17bea00640 1 -- 172.21.15.106:0/3390552024 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f17c0045500 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.915+0000 7f17dedf2740 10 client.25156 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.916+0000 7f17dedf2740 10 client.25156 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.916+0000 7f17dedf2740 20 client.25156 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.916+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55e7e1fcebf0 0x55e7e1fd0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.916+0000 7f17dedf2740 4 client.25156 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.916+0000 7f17dedf2740 1 -- 172.21.15.106:0/3390552024 --> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7e1f2a210 con 0x55e7e1fcebf0 2024-04-28T12:51:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.916+0000 7f17d9600640 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55e7e1fcebf0 0x55e7e1fd0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.917+0000 7f17bea00640 10 client.25156 ms_handle_connect on v2:172.21.15.171:6836/813345311 2024-04-28T12:51:04.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.916+0000 7f17d9600640 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55e7e1fcebf0 0x55e7e1fd0060 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a42d0f8083f7f23d server_cookie=d6a3b9d6536fe829 in_seq=0 out_seq=0 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.917+0000 7f17bea00640 1 -- 172.21.15.106:0/3390552024 <== mds.4 v2:172.21.15.171:6836/813345311 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7e1f2a210 con 0x55e7e1fcebf0 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.917+0000 7f17bea00640 10 client.25156 handle_command_reply: tid=0 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 client.25156 shutdown 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 -- 172.21.15.106:0/3390552024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 msgr2=0x55e7e1fc2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc2a20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005ab0 tx=0x7f17c00044e0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 -- 172.21.15.106:0/3390552024 shutdown_connections 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55e7e1fcebf0 0x55e7e1fd0060 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc2a20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17bea00640 0 client.25156 ms_handle_reset on v2:172.21.15.171:6836/813345311 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3390552024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 -- 172.21.15.106:0/3390552024 >> 172.21.15.106:0/3390552024 conn(0x55e7e1f4cd10 msgr2=0x55e7e1fb2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 -- 172.21.15.106:0/3390552024 shutdown_connections 2024-04-28T12:51:04.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 1 -- 172.21.15.106:0/3390552024 wait complete. 2024-04-28T12:51:04.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.918+0000 7f17dedf2740 20 client.25156 trim_cache size 0 max 16384 2024-04-28T12:51:04.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.938+0000 7f17dedf2740 1 Processor -- start 2024-04-28T12:51:04.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.938+0000 7f17dedf2740 1 -- start start 2024-04-28T12:51:04.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fb2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17dedf2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1fce3a0 con 0x55e7e1fb80f0 2024-04-28T12:51:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17dedf2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1fcea70 con 0x55e7e1fbdc50 2024-04-28T12:51:04.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17dedf2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1fcf140 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36264/0 (socket says 172.21.15.106:36264) 2024-04-28T12:51:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d9600640 1 -- 172.21.15.106:0/481690844 learned_addr learned my addr 172.21.15.106:0/481690844 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d8c00640 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17da000640 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fb2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d9600640 1 -- 172.21.15.106:0/481690844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 msgr2=0x55e7e1fc9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d9600640 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d9600640 1 -- 172.21.15.106:0/481690844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 msgr2=0x55e7e1fb2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d9600640 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fb2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d9600640 1 -- 172.21.15.106:0/481690844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e1fb8ec0 con 0x55e7e1fb80f0 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17da000640 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fb2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.939+0000 7f17d8c00640 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.940+0000 7f17d9600640 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb2620 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17c40d47f0 tx=0x7f17c40bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61789aa063fe3e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.940+0000 7f17c9c00640 1 -- 172.21.15.106:0/481690844 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c409caa0 con 0x55e7e1fb80f0 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.940+0000 7f17c9c00640 1 -- 172.21.15.106:0/481690844 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c4009820 con 0x55e7e1fb80f0 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.940+0000 7f17c9c00640 1 -- 172.21.15.106:0/481690844 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c409c630 con 0x55e7e1fb80f0 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 -- 172.21.15.106:0/481690844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 msgr2=0x55e7e1fb2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb2620 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f17c40d47f0 tx=0x7f17c40bd040 comp rx=0 tx=0).stop 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 -- 172.21.15.106:0/481690844 shutdown_connections 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fb2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 --2- 172.21.15.106:0/481690844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fb2620 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 -- 172.21.15.106:0/481690844 >> 172.21.15.106:0/481690844 conn(0x55e7e1f4cd10 msgr2=0x55e7e1fc10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 -- 172.21.15.106:0/481690844 shutdown_connections 2024-04-28T12:51:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 -- 172.21.15.106:0/481690844 wait complete. 2024-04-28T12:51:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 -- start start 2024-04-28T12:51:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.941+0000 7f17dedf2740 1 Processor -- start 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17dedf2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17dedf2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1fcfee0 con 0x55e7e1fb80f0 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17dedf2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1fcf810 con 0x55e7e1fbdc50 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17dedf2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e7e1fd05b0 con 0x55e7e1fb8ae0 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17dedf2740 10 client.0 _ll_register_callbacks cb 0x7f17c40d4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17d9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17d9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36278/0 (socket says 172.21.15.106:36278) 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17d9600640 1 -- 172.21.15.106:0/3962935304 learned_addr learned my addr 172.21.15.106:0/3962935304 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17d8c00640 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17da000640 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17d9600640 1 -- 172.21.15.106:0/3962935304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 msgr2=0x55e7e1fc6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17d9600640 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17d9600640 1 -- 172.21.15.106:0/3962935304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 msgr2=0x55e7e1fc9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17d9600640 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17d9600640 1 -- 172.21.15.106:0/3962935304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e1fc9750 con 0x55e7e1fb80f0 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.942+0000 7f17da000640 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.943+0000 7f17d9600640 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc5c00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17c40996a0 tx=0x7f17c4004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50dca02245ed59b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.943+0000 7f17bea00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.943+0000 7f17bea00640 1 -- 172.21.15.106:0/3962935304 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c409c810 con 0x55e7e1fb80f0 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.943+0000 7f17bea00640 1 -- 172.21.15.106:0/3962935304 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f17c409afd0 con 0x55e7e1fb80f0 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.943+0000 7f17dedf2740 1 -- 172.21.15.106:0/3962935304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c4009820 con 0x55e7e1fb80f0 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.943+0000 7f17bea00640 1 -- 172.21.15.106:0/3962935304 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17c409c630 con 0x55e7e1fb80f0 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.943+0000 7f17dedf2740 10 client.15060 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:04.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.943+0000 7f17dedf2740 1 -- 172.21.15.106:0/3962935304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7e203deb0 con 0x55e7e1fb80f0 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.943+0000 7f17bea00640 1 -- 172.21.15.106:0/3962935304 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17c4009820 con 0x55e7e1fb80f0 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.944+0000 7f17bea00640 1 -- 172.21.15.106:0/3962935304 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f17c4099340 con 0x55e7e1fb80f0 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.944+0000 7f17dedf2740 10 client.15060 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.944+0000 7f17dedf2740 10 client.15060 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.944+0000 7f17dedf2740 20 client.15060 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.944+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55e7e2042520 0x55e7e2041230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.944+0000 7f17dedf2740 4 client.15060 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.944+0000 7f17dedf2740 1 -- 172.21.15.106:0/3962935304 --> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] -- command(tid 0: {"prefix": "status"}) -- 0x55e7e1f24080 con 0x55e7e2042520 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.944+0000 7f17d8c00640 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55e7e2042520 0x55e7e2041230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17d8c00640 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55e7e2042520 0x55e7e2041230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4b2ca96d95ced41 server_cookie=bfbf505b536eded3 in_seq=0 out_seq=0 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17bea00640 10 client.15060 ms_handle_connect on v2:172.21.15.171:6836/813345311 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17bea00640 1 -- 172.21.15.106:0/3962935304 <== mds.4 v2:172.21.15.171:6836/813345311 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55e7e1f24080 con 0x55e7e2042520 2024-04-28T12:51:04.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17bea00640 10 client.15060 handle_command_reply: tid=0 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 client.15060 shutdown 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 -- 172.21.15.106:0/3962935304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 msgr2=0x55e7e1fc5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc5c00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f17c40996a0 tx=0x7f17c4004930 comp rx=0 tx=0).stop 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 -- 172.21.15.106:0/3962935304 shutdown_connections 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55e7e2042520 0x55e7e2041230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e1fbdc50 0x55e7e1fc9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17bea00640 0 client.15060 ms_handle_reset on v2:172.21.15.171:6836/813345311 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e7e1fb8ae0 0x55e7e1fc6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 --2- 172.21.15.106:0/3962935304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7e1fb80f0 0x55e7e1fc5c00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 -- 172.21.15.106:0/3962935304 >> 172.21.15.106:0/3962935304 conn(0x55e7e1f4cd10 msgr2=0x55e7e1fb2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 -- 172.21.15.106:0/3962935304 shutdown_connections 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 1 -- 172.21.15.106:0/3962935304 wait complete. 2024-04-28T12:51:04.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.945+0000 7f17dedf2740 20 client.15060 trim_cache size 0 max 16384 2024-04-28T12:51:04.948 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:04.948 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:04.948 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 4, 2024-04-28T12:51:04.948 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24520, 2024-04-28T12:51:04.948 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:04.948 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:04.948 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:04.948 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 95.929996273, 2024-04-28T12:51:04.948 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:04.949 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:04.949 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:04.949 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 152.96291198099999 2024-04-28T12:51:04.949 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:04.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 -- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17b80a3c40 msgr2=0x7f17b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17b80a3c40 0x7f17b80a6100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55e7e1fb9340 tx=0x7f17cc005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 -- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 msgr2=0x7f17d419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 0x7f17d419ae10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009d50 tx=0x7f17c400c6a0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 -- 172.21.15.106:0/3756569512 shutdown_connections 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17b80a3c40 0x7f17b80a6100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17d410a840 0x7f17d41a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17d4076970 0x7f17d419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 --2- 172.21.15.106:0/3756569512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17d4075390 0x7f17d419ae10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 -- 172.21.15.106:0/3756569512 >> 172.21.15.106:0/3756569512 conn(0x7f17d40fdf10 msgr2=0x7f17d4106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 -- 172.21.15.106:0/3756569512 shutdown_connections 2024-04-28T12:51:04.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:04.946+0000 7f17dc000640 1 -- 172.21.15.106:0/3756569512 wait complete. 2024-04-28T12:51:04.960 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-28T12:51:05.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.160+0000 7f2058a00640 1 -- 172.21.15.106:0/2335203957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2054102850 msgr2=0x7f2054102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.160+0000 7f2058a00640 1 --2- 172.21.15.106:0/2335203957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2054102850 0x7f2054102cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f203c009ca0 tx=0x7f203c02a8c0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.160+0000 7f2058a00640 1 -- 172.21.15.106:0/2335203957 shutdown_connections 2024-04-28T12:51:05.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.160+0000 7f2058a00640 1 --2- 172.21.15.106:0/2335203957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2054103210 0x7f2054109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.160+0000 7f2058a00640 1 --2- 172.21.15.106:0/2335203957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2054102850 0x7f2054102cd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.160+0000 7f2058a00640 1 --2- 172.21.15.106:0/2335203957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2054101660 0x7f2054101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.160+0000 7f2058a00640 1 -- 172.21.15.106:0/2335203957 >> 172.21.15.106:0/2335203957 conn(0x7f20540fcbb0 msgr2=0x7f20540ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.160+0000 7f2058a00640 1 -- 172.21.15.106:0/2335203957 shutdown_connections 2024-04-28T12:51:05.158 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.160+0000 7f2058a00640 1 -- 172.21.15.106:0/2335203957 wait complete. 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.161+0000 7f2058a00640 1 Processor -- start 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.161+0000 7f2058a00640 1 -- start start 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.161+0000 7f2058a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2054101660 0x7f20541989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.161+0000 7f2058a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2054102850 0x7f2054198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.161+0000 7f2058a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2054103210 0x7f205419ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.161+0000 7f2058a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f205410e2f0 con 0x7f2054101660 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.161+0000 7f2058a00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f205410e170 con 0x7f2054103210 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.161+0000 7f2058a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f205410e470 con 0x7f2054102850 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2052800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2054101660 0x7f20541989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2052800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2054101660 0x7f20541989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36286/0 (socket says 172.21.15.106:36286) 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2052800640 1 -- 172.21.15.106:0/18131161 learned_addr learned my addr 172.21.15.106:0/18131161 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2051e00640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2054102850 0x7f2054198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2053200640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2054103210 0x7f205419ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2052800640 1 -- 172.21.15.106:0/18131161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2054102850 msgr2=0x7f2054198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2052800640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2054102850 0x7f2054198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2052800640 1 -- 172.21.15.106:0/18131161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2054103210 msgr2=0x7f205419ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2052800640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2054103210 0x7f205419ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2052800640 1 -- 172.21.15.106:0/18131161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20541a05c0 con 0x7f2054101660 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.162+0000 7f2052800640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2054101660 0x7f20541989a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f204400e9c0 tx=0x7f204400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.163+0000 7f204b400640 1 -- 172.21.15.106:0/18131161 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204400ccc0 con 0x7f2054101660 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.163+0000 7f204b400640 1 -- 172.21.15.106:0/18131161 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20440035f0 con 0x7f2054101660 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.163+0000 7f2058a00640 1 -- 172.21.15.106:0/18131161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20541a08b0 con 0x7f2054101660 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.163+0000 7f2058a00640 1 -- 172.21.15.106:0/18131161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20541a0d80 con 0x7f2054101660 2024-04-28T12:51:05.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.163+0000 7f204b400640 1 -- 172.21.15.106:0/18131161 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204400ce60 con 0x7f2054101660 2024-04-28T12:51:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.166+0000 7f205c21b740 1 Processor -- start 2024-04-28T12:51:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f204b400640 1 -- 172.21.15.106:0/18131161 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2044004000 con 0x7f2054101660 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f205c21b740 1 -- start start 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f205c21b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5456d0 0x561adc545af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f205c21b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc54abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f205c21b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc54d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f205c21b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561adc4a5600 con 0x561adc54b230 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f205c21b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561adc442100 con 0x561adc5460c0 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f205c21b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561adc4a15c0 con 0x561adc5456d0 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f2052800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc54abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f2053200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5456d0 0x561adc545af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f2053200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5456d0 0x561adc545af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:47882/0 (socket says 172.21.15.106:47882) 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f2052800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc54abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:33030/0 (socket says 172.21.15.106:33030) 2024-04-28T12:51:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.167+0000 7f2053200640 1 -- 172.21.15.106:0/3734196672 learned_addr learned my addr 172.21.15.106:0/3734196672 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2051e00640 1 --2- 172.21.15.106:0/3734196672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc54d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2053200640 1 -- 172.21.15.106:0/3734196672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 msgr2=0x561adc54abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2053200640 1 --2- 172.21.15.106:0/3734196672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc54abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2053200640 1 -- 172.21.15.106:0/3734196672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 msgr2=0x561adc54d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2053200640 1 --2- 172.21.15.106:0/3734196672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc54d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2053200640 1 -- 172.21.15.106:0/3734196672 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561adc1b72d0 con 0x561adc5456d0 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2051e00640 1 --2- 172.21.15.106:0/3734196672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc54d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2053200640 1 --2- 172.21.15.106:0/3734196672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5456d0 0x561adc545af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f204c009c90 tx=0x7f204c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47472b2485a775c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f204b400640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f20340a3c40 0x7f20340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2043400640 1 -- 172.21.15.106:0/3734196672 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c004230 con 0x561adc5456d0 2024-04-28T12:51:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2043400640 1 -- 172.21.15.106:0/3734196672 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f204c009e40 con 0x561adc5456d0 2024-04-28T12:51:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.168+0000 7f2043400640 1 -- 172.21.15.106:0/3734196672 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c019760 con 0x561adc5456d0 2024-04-28T12:51:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f2052800640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f20340a3c40 0x7f20340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f204b400640 1 -- 172.21.15.106:0/18131161 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f20440d0e20 con 0x7f2054101660 2024-04-28T12:51:05.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f205c21b740 1 -- 172.21.15.106:0/3734196672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5456d0 msgr2=0x561adc545af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f2052800640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f20340a3c40 0x7f20340a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x561adc5464c0 tx=0x7f20440d4910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.172 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f205c21b740 1 --2- 172.21.15.106:0/3734196672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5456d0 0x561adc545af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f204c009c90 tx=0x7f204c011810 comp rx=0 tx=0).stop 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f205c21b740 1 -- 172.21.15.106:0/3734196672 shutdown_connections 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f205c21b740 1 --2- 172.21.15.106:0/3734196672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc54d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f205c21b740 1 --2- 172.21.15.106:0/3734196672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc54abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f205c21b740 1 --2- 172.21.15.106:0/3734196672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5456d0 0x561adc545af0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.169+0000 7f205c21b740 1 -- 172.21.15.106:0/3734196672 >> 172.21.15.106:0/3734196672 conn(0x561adc4d5d10 msgr2=0x561adc53cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.170+0000 7f205c21b740 1 -- 172.21.15.106:0/3734196672 shutdown_connections 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.170+0000 7f205c21b740 1 -- 172.21.15.106:0/3734196672 wait complete. 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.170+0000 7f205c21b740 1 -- start start 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.170+0000 7f205c21b740 1 Processor -- start 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f205c21b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 0x561adc54fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f205c21b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5460c0 0x561adc5500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f205c21b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc557110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f205c21b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561adc54e1e0 con 0x561adc54b230 2024-04-28T12:51:05.173 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f205c21b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561adc42cf70 con 0x561adc5456d0 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f205c21b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561adc54e360 con 0x561adc5460c0 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f205c21b740 10 client.0 _ll_register_callbacks cb 0x561adc49b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2053200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 0x561adc54fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2051e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5460c0 0x561adc5500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2053200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 0x561adc54fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:33046/0 (socket says 172.21.15.106:33046) 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2053200640 1 -- 172.21.15.106:0/2222036951 learned_addr learned my addr 172.21.15.106:0/2222036951 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2053200640 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc557110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2051e00640 1 -- 172.21.15.106:0/2222036951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 msgr2=0x561adc54fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2051e00640 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 0x561adc54fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2051e00640 1 -- 172.21.15.106:0/2222036951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 msgr2=0x561adc557110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2051e00640 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc557110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2051e00640 1 -- 172.21.15.106:0/2222036951 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561adc2816b0 con 0x561adc5460c0 2024-04-28T12:51:05.174 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.171+0000 7f2053200640 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc557110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.172+0000 7f2051e00640 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5460c0 0x561adc5500d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f203c02aeb0 tx=0x7f203c005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5119f5e3ba8e846 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.172+0000 7f203be00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.172+0000 7f203be00640 1 -- 172.21.15.106:0/2222036951 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c003d80 con 0x561adc5460c0 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.172+0000 7f205c21b740 1 -- 172.21.15.106:0/2222036951 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561adc4b3210 con 0x561adc5460c0 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.172+0000 7f203be00640 1 -- 172.21.15.106:0/2222036951 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f203c004040 con 0x561adc5460c0 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.172+0000 7f203be00640 1 -- 172.21.15.106:0/2222036951 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f203c004240 con 0x561adc5460c0 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.172+0000 7f205c21b740 10 client.25174 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.172+0000 7f205c21b740 1 -- 172.21.15.106:0/2222036951 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561adc55a060 con 0x561adc5460c0 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.172+0000 7f203be00640 1 -- 172.21.15.106:0/2222036951 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561adc4b3210 con 0x561adc5460c0 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.173+0000 7f203be00640 1 -- 172.21.15.106:0/2222036951 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f203c02fbb0 con 0x561adc5460c0 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.173+0000 7f205c21b740 10 client.25174 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.173+0000 7f205c21b740 10 client.25174 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.173+0000 7f205c21b740 20 client.25174 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.173+0000 7f205c21b740 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x561adc55c6d0 0x561adc55db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.173+0000 7f205c21b740 4 client.25174 mds_command: new command op to 24538 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.173+0000 7f205c21b740 1 -- 172.21.15.106:0/2222036951 --> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561adc4ad080 con 0x561adc55c6d0 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.173+0000 7f2053200640 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x561adc55c6d0 0x561adc55db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.174+0000 7f2053200640 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x561adc55c6d0 0x561adc55db40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1f5068f533217b6 server_cookie=4190ae537d08687d in_seq=0 out_seq=0 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.174+0000 7f203be00640 10 client.25174 ms_handle_connect on v2:172.21.15.171:6838/2406303763 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f203be00640 1 -- 172.21.15.106:0/2222036951 <== mds.3 v2:172.21.15.171:6838/2406303763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561adc4ad080 con 0x561adc55c6d0 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f203be00640 10 client.25174 handle_command_reply: tid=0 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 client.25174 shutdown 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 -- 172.21.15.106:0/2222036951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5460c0 msgr2=0x561adc5500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5460c0 0x561adc5500d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f203c02aeb0 tx=0x7f203c005ef0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 -- 172.21.15.106:0/2222036951 shutdown_connections 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x561adc55c6d0 0x561adc55db40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc54b230 0x561adc557110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc5460c0 0x561adc5500d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 --2- 172.21.15.106:0/2222036951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 0x561adc54fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f203be00640 0 client.25174 ms_handle_reset on v2:172.21.15.171:6838/2406303763 2024-04-28T12:51:05.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 -- 172.21.15.106:0/2222036951 >> 172.21.15.106:0/2222036951 conn(0x561adc4d5d10 msgr2=0x561adc53bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 -- 172.21.15.106:0/2222036951 shutdown_connections 2024-04-28T12:51:05.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 1 -- 172.21.15.106:0/2222036951 wait complete. 2024-04-28T12:51:05.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.175+0000 7f205c21b740 20 client.25174 trim_cache size 0 max 16384 2024-04-28T12:51:05.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.195+0000 7f205c21b740 1 Processor -- start 2024-04-28T12:51:05.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.195+0000 7f205c21b740 1 -- start start 2024-04-28T12:51:05.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.195+0000 7f205c21b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 0x561adc547520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.195+0000 7f205c21b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5460c0 0x561adc5570d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.195+0000 7f205c21b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc54b230 0x561adc547a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.195+0000 7f205c21b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561adc55be80 con 0x561adc5460c0 2024-04-28T12:51:05.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.195+0000 7f205c21b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561adc55c550 con 0x561adc5456d0 2024-04-28T12:51:05.192 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.195+0000 7f205c21b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561adc55cc20 con 0x561adc54b230 2024-04-28T12:51:05.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2051e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5460c0 0x561adc5570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.193 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2051e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5460c0 0x561adc5570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36302/0 (socket says 172.21.15.106:36302) 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2051e00640 1 -- 172.21.15.106:0/177508782 learned_addr learned my addr 172.21.15.106:0/177508782 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2053200640 1 --2- 172.21.15.106:0/177508782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 0x561adc547520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2053200640 1 --2- 172.21.15.106:0/177508782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc54b230 0x561adc547a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2051e00640 1 -- 172.21.15.106:0/177508782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc54b230 msgr2=0x561adc547a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2051e00640 1 --2- 172.21.15.106:0/177508782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc54b230 0x561adc547a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2051e00640 1 -- 172.21.15.106:0/177508782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 msgr2=0x561adc547520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2051e00640 1 --2- 172.21.15.106:0/177508782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 0x561adc547520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2051e00640 1 -- 172.21.15.106:0/177508782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561adc55bca0 con 0x561adc5460c0 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.196+0000 7f2051e00640 1 --2- 172.21.15.106:0/177508782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5460c0 0x561adc5570d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f203c02f540 tx=0x7f203c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299deb97d739271 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f2042a00640 1 -- 172.21.15.106:0/177508782 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c004b40 con 0x561adc5460c0 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f2042a00640 1 -- 172.21.15.106:0/177508782 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203c036040 con 0x561adc5460c0 2024-04-28T12:51:05.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f2042a00640 1 -- 172.21.15.106:0/177508782 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c004840 con 0x561adc5460c0 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f205c21b740 1 -- 172.21.15.106:0/177508782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5460c0 msgr2=0x561adc5570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f205c21b740 1 --2- 172.21.15.106:0/177508782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5460c0 0x561adc5570d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f203c02f540 tx=0x7f203c02a8c0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f205c21b740 1 -- 172.21.15.106:0/177508782 shutdown_connections 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f205c21b740 1 --2- 172.21.15.106:0/177508782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5460c0 0x561adc5570d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f205c21b740 1 --2- 172.21.15.106:0/177508782 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc54b230 0x561adc547a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f205c21b740 1 --2- 172.21.15.106:0/177508782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5456d0 0x561adc547520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f205c21b740 1 -- 172.21.15.106:0/177508782 >> 172.21.15.106:0/177508782 conn(0x561adc4d5d10 msgr2=0x561adc537df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f205c21b740 1 -- 172.21.15.106:0/177508782 shutdown_connections 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.197+0000 7f205c21b740 1 -- 172.21.15.106:0/177508782 wait complete. 2024-04-28T12:51:05.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.198+0000 7f205c21b740 1 -- start start 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.198+0000 7f205c21b740 1 Processor -- start 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.198+0000 7f205c21b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5456d0 0x561adc552ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.198+0000 7f205c21b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc552ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f205c21b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc54b230 0x561adc555f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f205c21b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561adc55d9c0 con 0x561adc5456d0 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f205c21b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561adc55d2f0 con 0x561adc5460c0 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f205c21b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561adc55e090 con 0x561adc54b230 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f205c21b740 10 client.0 _ll_register_callbacks cb 0x7f2044005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2053200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5456d0 0x561adc552ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2053200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5456d0 0x561adc552ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36318/0 (socket says 172.21.15.106:36318) 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2051e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc552ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2053200640 1 -- 172.21.15.106:0/3993255935 learned_addr learned my addr 172.21.15.106:0/3993255935 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2053200640 1 -- 172.21.15.106:0/3993255935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc54b230 msgr2=0x561adc555f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2053200640 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc54b230 0x561adc555f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2053200640 1 -- 172.21.15.106:0/3993255935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 msgr2=0x561adc552ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2053200640 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc552ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2053200640 1 -- 172.21.15.106:0/3993255935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561adc556540 con 0x561adc5456d0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2051e00640 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc552ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f2053200640 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5456d0 0x561adc552ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f204c011d10 tx=0x7f204c0043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533b066343561f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.199+0000 7f203be00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.200+0000 7f203be00640 1 -- 172.21.15.106:0/3993255935 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c03b060 con 0x561adc5456d0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.200+0000 7f203be00640 1 -- 172.21.15.106:0/3993255935 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f204c01f040 con 0x561adc5456d0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.200+0000 7f205c21b740 1 -- 172.21.15.106:0/3993255935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f203c036040 con 0x561adc5456d0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.200+0000 7f203be00640 1 -- 172.21.15.106:0/3993255935 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f204c042050 con 0x561adc5456d0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.200+0000 7f205c21b740 10 client.15075 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.200+0000 7f205c21b740 1 -- 172.21.15.106:0/3993255935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561adc5c28b0 con 0x561adc5456d0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.200+0000 7f203be00640 1 -- 172.21.15.106:0/3993255935 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203c036040 con 0x561adc5456d0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.201+0000 7f203be00640 1 -- 172.21.15.106:0/3993255935 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f204c025040 con 0x561adc5456d0 2024-04-28T12:51:05.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.201+0000 7f205c21b740 10 client.15075 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:05.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.201+0000 7f205c21b740 10 client.15075 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-28T12:51:05.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.201+0000 7f205c21b740 20 client.15075 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:05.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.201+0000 7f205c21b740 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x561adc5c72f0 0x561adc5c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.201+0000 7f205c21b740 4 client.15075 mds_command: new command op to 24538 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:05.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.201+0000 7f205c21b740 1 -- 172.21.15.106:0/3993255935 --> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] -- command(tid 0: {"prefix": "status"}) -- 0x561adc428860 con 0x561adc5c72f0 2024-04-28T12:51:05.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.201+0000 7f2051e00640 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x561adc5c72f0 0x561adc5c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f2051e00640 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x561adc5c72f0 0x561adc5c6000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a0ce31abff0811db server_cookie=a96e50976d238d47 in_seq=0 out_seq=0 2024-04-28T12:51:05.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f203be00640 10 client.15075 ms_handle_connect on v2:172.21.15.171:6838/2406303763 2024-04-28T12:51:05.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f203be00640 1 -- 172.21.15.106:0/3993255935 <== mds.3 v2:172.21.15.171:6838/2406303763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561adc428860 con 0x561adc5c72f0 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f203be00640 10 client.15075 handle_command_reply: tid=0 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 client.15075 shutdown 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 -- 172.21.15.106:0/3993255935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5456d0 msgr2=0x561adc552ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5456d0 0x561adc552ab0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f204c011d10 tx=0x7f204c0043e0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 -- 172.21.15.106:0/3993255935 shutdown_connections 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x561adc5c72f0 0x561adc5c6000 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561adc54b230 0x561adc555f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561adc5460c0 0x561adc552ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f203be00640 0 client.15075 ms_handle_reset on v2:172.21.15.171:6838/2406303763 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 --2- 172.21.15.106:0/3993255935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561adc5456d0 0x561adc552ab0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 -- 172.21.15.106:0/3993255935 >> 172.21.15.106:0/3993255935 conn(0x561adc4d5d10 msgr2=0x561adc547520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.202+0000 7f205c21b740 1 -- 172.21.15.106:0/3993255935 shutdown_connections 2024-04-28T12:51:05.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f205c21b740 1 -- 172.21.15.106:0/3993255935 wait complete. 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f205c21b740 20 client.15075 trim_cache size 0 max 16384 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 3, 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24538, 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 97.206544788000002, 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:05.204 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:05.205 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 147.317787651 2024-04-28T12:51:05.205 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:05.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 -- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f20340a3c40 msgr2=0x7f20340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f20340a3c40 0x7f20340a6100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x561adc5464c0 tx=0x7f20440d4910 comp rx=0 tx=0).stop 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 -- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2054101660 msgr2=0x7f20541989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2054101660 0x7f20541989a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f204400e9c0 tx=0x7f204400edc0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 -- 172.21.15.106:0/18131161 shutdown_connections 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f20340a3c40 0x7f20340a6100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2054103210 0x7f205419ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2054102850 0x7f2054198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 --2- 172.21.15.106:0/18131161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2054101660 0x7f20541989a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 -- 172.21.15.106:0/18131161 >> 172.21.15.106:0/18131161 conn(0x7f20540fcbb0 msgr2=0x7f2054105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.203+0000 7f2058a00640 1 -- 172.21.15.106:0/18131161 shutdown_connections 2024-04-28T12:51:05.206 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.204+0000 7f2058a00640 1 -- 172.21.15.106:0/18131161 wait complete. 2024-04-28T12:51:05.220 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-28T12:51:05.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.417+0000 7f475d400640 1 -- 172.21.15.106:0/3774387440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758106760 msgr2=0x7f475810cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.417+0000 7f475d400640 1 --2- 172.21.15.106:0/3774387440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758106760 0x7f475810cff0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4750009ca0 tx=0x7f475002a8b0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.417+0000 7f475d400640 1 -- 172.21.15.106:0/3774387440 shutdown_connections 2024-04-28T12:51:05.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.417+0000 7f475d400640 1 --2- 172.21.15.106:0/3774387440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4758105da0 0x7f4758106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.417+0000 7f475d400640 1 --2- 172.21.15.106:0/3774387440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758106760 0x7f475810cff0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.417+0000 7f475d400640 1 --2- 172.21.15.106:0/3774387440 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4758104bb0 0x7f4758104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.417+0000 7f475d400640 1 -- 172.21.15.106:0/3774387440 >> 172.21.15.106:0/3774387440 conn(0x7f4758100120 msgr2=0x7f4758102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.417+0000 7f475d400640 1 -- 172.21.15.106:0/3774387440 shutdown_connections 2024-04-28T12:51:05.415 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.417+0000 7f475d400640 1 -- 172.21.15.106:0/3774387440 wait complete. 2024-04-28T12:51:05.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.418+0000 7f475d400640 1 Processor -- start 2024-04-28T12:51:05.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.418+0000 7f475d400640 1 -- start start 2024-04-28T12:51:05.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f475d400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4758104bb0 0x7f475819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f475d400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758105da0 0x7f475819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f475d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4758106760 0x7f47581a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f475d400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f47581106e0 con 0x7f4758106760 2024-04-28T12:51:05.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f475d400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4758110560 con 0x7f4758105da0 2024-04-28T12:51:05.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f475d400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4758110860 con 0x7f4758104bb0 2024-04-28T12:51:05.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4757200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4758106760 0x7f47581a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4755e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758105da0 0x7f475819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4756800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4758104bb0 0x7f475819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4757200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4758106760 0x7f47581a2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36332/0 (socket says 172.21.15.106:36332) 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4757200640 1 -- 172.21.15.106:0/860817508 learned_addr learned my addr 172.21.15.106:0/860817508 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4757200640 1 -- 172.21.15.106:0/860817508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4758104bb0 msgr2=0x7f475819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4757200640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4758104bb0 0x7f475819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4757200640 1 -- 172.21.15.106:0/860817508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758105da0 msgr2=0x7f475819b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4757200640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758105da0 0x7f475819b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.419+0000 7f4757200640 1 -- 172.21.15.106:0/860817508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47581a2990 con 0x7f4758106760 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.420+0000 7f4757200640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4758106760 0x7f47581a2320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4750005e30 tx=0x7f4750003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.420+0000 7f474f400640 1 -- 172.21.15.106:0/860817508 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750004240 con 0x7f4758106760 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.420+0000 7f474f400640 1 -- 172.21.15.106:0/860817508 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4750040070 con 0x7f4758106760 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.420+0000 7f475d400640 1 -- 172.21.15.106:0/860817508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47581a2c20 con 0x7f4758106760 2024-04-28T12:51:05.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.420+0000 7f474f400640 1 -- 172.21.15.106:0/860817508 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750045bd0 con 0x7f4758106760 2024-04-28T12:51:05.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.420+0000 7f475d400640 1 -- 172.21.15.106:0/860817508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47581a3170 con 0x7f4758106760 2024-04-28T12:51:05.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.423+0000 7f476020c740 1 Processor -- start 2024-04-28T12:51:05.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.423+0000 7f474f400640 1 -- 172.21.15.106:0/860817508 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f47500043e0 con 0x7f4758106760 2024-04-28T12:51:05.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f476020c740 1 -- start start 2024-04-28T12:51:05.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f476020c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b28510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f476020c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b28ae0 0x5627a2b2d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f476020c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 0x5627a2b300e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f476020c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5627a2a8c600 con 0x5627a2b28ae0 2024-04-28T12:51:05.422 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f476020c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5627a2a29100 con 0x5627a2b2dc50 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f476020c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5627a2a885c0 con 0x5627a2b280f0 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f4756800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b28ae0 0x5627a2b2d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f4755e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b28510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f4756800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b28ae0 0x5627a2b2d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36338/0 (socket says 172.21.15.106:36338) 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f4756800640 1 -- 172.21.15.106:0/282801768 learned_addr learned my addr 172.21.15.106:0/282801768 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.424+0000 7f4755e00640 1 --2- 172.21.15.106:0/282801768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 0x5627a2b300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4756800640 1 -- 172.21.15.106:0/282801768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 msgr2=0x5627a2b28510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4756800640 1 --2- 172.21.15.106:0/282801768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b28510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4756800640 1 -- 172.21.15.106:0/282801768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 msgr2=0x5627a2b300e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4756800640 1 --2- 172.21.15.106:0/282801768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 0x5627a2b300e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4756800640 1 -- 172.21.15.106:0/282801768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a279e2d0 con 0x5627a2b28ae0 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4755e00640 1 --2- 172.21.15.106:0/282801768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 0x5627a2b300e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f474f400640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f473c0a3c40 0x7f473c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4756800640 1 --2- 172.21.15.106:0/282801768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b28ae0 0x5627a2b2d5e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4748009ca0 tx=0x7f4748011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531a1006dbbd92dd server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.423 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f474f400640 1 -- 172.21.15.106:0/860817508 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f47500f1a40 con 0x7f4758106760 2024-04-28T12:51:05.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4755e00640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f473c0a3c40 0x7f473c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4747400640 1 -- 172.21.15.106:0/282801768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748004220 con 0x5627a2b28ae0 2024-04-28T12:51:05.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4747400640 1 -- 172.21.15.106:0/282801768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47480043c0 con 0x5627a2b28ae0 2024-04-28T12:51:05.424 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.425+0000 7f4747400640 1 -- 172.21.15.106:0/282801768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47480197f0 con 0x5627a2b28ae0 2024-04-28T12:51:05.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f476020c740 1 -- 172.21.15.106:0/282801768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b28ae0 msgr2=0x5627a2b2d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.427 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f476020c740 1 --2- 172.21.15.106:0/282801768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b28ae0 0x5627a2b2d5e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4748009ca0 tx=0x7f4748011810 comp rx=0 tx=0).stop 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f4755e00640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f473c0a3c40 0x7f473c0a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5627a2b29340 tx=0x7f4740013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f476020c740 1 -- 172.21.15.106:0/282801768 shutdown_connections 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f476020c740 1 --2- 172.21.15.106:0/282801768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b28ae0 0x5627a2b2d5e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f476020c740 1 --2- 172.21.15.106:0/282801768 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 0x5627a2b300e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f476020c740 1 --2- 172.21.15.106:0/282801768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b28510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f476020c740 1 -- 172.21.15.106:0/282801768 >> 172.21.15.106:0/282801768 conn(0x5627a2abcd10 msgr2=0x5627a2b23b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f476020c740 1 -- 172.21.15.106:0/282801768 shutdown_connections 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.426+0000 7f476020c740 1 -- 172.21.15.106:0/282801768 wait complete. 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.427+0000 7f476020c740 1 -- start start 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.427+0000 7f476020c740 1 Processor -- start 2024-04-28T12:51:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.427+0000 7f476020c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f476020c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 0x5627a2b32ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f476020c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b2dc50 0x5627a2b39b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f476020c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5627a2b30c00 con 0x5627a2b2dc50 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f476020c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5627a2a13f70 con 0x5627a2b28ae0 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f476020c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5627a2b30d80 con 0x5627a2b280f0 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f476020c740 10 client.0 _ll_register_callbacks cb 0x5627a2a82480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4756800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4757200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b2dc50 0x5627a2b39b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4755e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 0x5627a2b32ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4756800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b32580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:47940/0 (socket says 172.21.15.106:47940) 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4756800640 1 -- 172.21.15.106:0/61757633 learned_addr learned my addr 172.21.15.106:0/61757633 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4756800640 1 -- 172.21.15.106:0/61757633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 msgr2=0x5627a2b32ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4756800640 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 0x5627a2b32ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4756800640 1 -- 172.21.15.106:0/61757633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b2dc50 msgr2=0x5627a2b39b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4756800640 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b2dc50 0x5627a2b39b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4756800640 1 -- 172.21.15.106:0/61757633 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a28686b0 con 0x5627a2b280f0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.428+0000 7f4757200640 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b2dc50 0x5627a2b39b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.429+0000 7f4756800640 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b32580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4748009c70 tx=0x7f47480052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e36fccabf6bd59 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.429+0000 7f473be00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.429+0000 7f473be00640 1 -- 172.21.15.106:0/61757633 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748028070 con 0x5627a2b280f0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.429+0000 7f476020c740 1 -- 172.21.15.106:0/61757633 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47480043c0 con 0x5627a2b280f0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.429+0000 7f473be00640 1 -- 172.21.15.106:0/61757633 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f47480035f0 con 0x5627a2b280f0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.429+0000 7f473be00640 1 -- 172.21.15.106:0/61757633 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4748034050 con 0x5627a2b280f0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.429+0000 7f476020c740 10 client.25189 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.429+0000 7f476020c740 1 -- 172.21.15.106:0/61757633 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627a2b3cb30 con 0x5627a2b280f0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.429+0000 7f473be00640 1 -- 172.21.15.106:0/61757633 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47480043c0 con 0x5627a2b280f0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.430+0000 7f473be00640 1 -- 172.21.15.106:0/61757633 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4748004a70 con 0x5627a2b280f0 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.430+0000 7f476020c740 10 client.25189 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.430+0000 7f476020c740 10 client.25189 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-28T12:51:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.430+0000 7f476020c740 20 client.25189 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.430+0000 7f476020c740 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x5627a2b3efd0 0x5627a2b40440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.430+0000 7f476020c740 4 client.25189 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.430+0000 7f476020c740 1 -- 172.21.15.106:0/61757633 --> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627a2a9a210 con 0x5627a2b3efd0 2024-04-28T12:51:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.430+0000 7f4755e00640 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x5627a2b3efd0 0x5627a2b40440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.431+0000 7f4755e00640 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x5627a2b3efd0 0x5627a2b40440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1a4e9c1bd4da215 server_cookie=546b0c9fae68e371 in_seq=0 out_seq=0 2024-04-28T12:51:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.431+0000 7f473be00640 10 client.25189 ms_handle_connect on v2:172.21.15.106:6838/3863798001 2024-04-28T12:51:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f473be00640 1 -- 172.21.15.106:0/61757633 <== mds.2 v2:172.21.15.106:6838/3863798001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5627a2a9a210 con 0x5627a2b3efd0 2024-04-28T12:51:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f473be00640 10 client.25189 handle_command_reply: tid=0 2024-04-28T12:51:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 client.25189 shutdown 2024-04-28T12:51:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 -- 172.21.15.106:0/61757633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 msgr2=0x5627a2b32580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b32580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4748009c70 tx=0x7f47480052f0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 -- 172.21.15.106:0/61757633 shutdown_connections 2024-04-28T12:51:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x5627a2b3efd0 0x5627a2b40440 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b2dc50 0x5627a2b39b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 0x5627a2b32ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f473be00640 0 client.25189 ms_handle_reset on v2:172.21.15.106:6838/3863798001 2024-04-28T12:51:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 --2- 172.21.15.106:0/61757633 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b280f0 0x5627a2b32580 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 -- 172.21.15.106:0/61757633 >> 172.21.15.106:0/61757633 conn(0x5627a2abcd10 msgr2=0x5627a2b22eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 -- 172.21.15.106:0/61757633 shutdown_connections 2024-04-28T12:51:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 1 -- 172.21.15.106:0/61757633 wait complete. 2024-04-28T12:51:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.432+0000 7f476020c740 20 client.25189 trim_cache size 0 max 16384 2024-04-28T12:51:05.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.452+0000 7f476020c740 1 Processor -- start 2024-04-28T12:51:05.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.452+0000 7f476020c740 1 -- start start 2024-04-28T12:51:05.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f476020c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b22620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f476020c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b28ae0 0x5627a2b39a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f476020c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 0x5627a2b22b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f476020c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5627a2b3e780 con 0x5627a2b280f0 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f476020c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5627a2b3ee50 con 0x5627a2b2dc50 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f476020c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5627a2b3f520 con 0x5627a2b28ae0 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4756800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b22620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4756800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b22620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36356/0 (socket says 172.21.15.106:36356) 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4756800640 1 -- 172.21.15.106:0/769010523 learned_addr learned my addr 172.21.15.106:0/769010523 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4757200640 1 --2- 172.21.15.106:0/769010523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 0x5627a2b22b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4755e00640 1 --2- 172.21.15.106:0/769010523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b28ae0 0x5627a2b39a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4756800640 1 -- 172.21.15.106:0/769010523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b28ae0 msgr2=0x5627a2b39a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4756800640 1 --2- 172.21.15.106:0/769010523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b28ae0 0x5627a2b39a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4756800640 1 -- 172.21.15.106:0/769010523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 msgr2=0x5627a2b22b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4756800640 1 --2- 172.21.15.106:0/769010523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 0x5627a2b22b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.453+0000 7f4756800640 1 -- 172.21.15.106:0/769010523 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a2b1f8a0 con 0x5627a2b280f0 2024-04-28T12:51:05.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.454+0000 7f4756800640 1 --2- 172.21.15.106:0/769010523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b22620 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4748001eb0 tx=0x7f4748009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9365582c776ca519 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.454+0000 7f4746a00640 1 -- 172.21.15.106:0/769010523 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4748028040 con 0x5627a2b280f0 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.454+0000 7f4746a00640 1 -- 172.21.15.106:0/769010523 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4748003aa0 con 0x5627a2b280f0 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.454+0000 7f4746a00640 1 -- 172.21.15.106:0/769010523 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47480199a0 con 0x5627a2b280f0 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.454+0000 7f476020c740 1 -- 172.21.15.106:0/769010523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 msgr2=0x5627a2b22620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.454+0000 7f476020c740 1 --2- 172.21.15.106:0/769010523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b22620 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4748001eb0 tx=0x7f4748009b10 comp rx=0 tx=0).stop 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.455+0000 7f476020c740 1 -- 172.21.15.106:0/769010523 shutdown_connections 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.455+0000 7f476020c740 1 --2- 172.21.15.106:0/769010523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b2dc50 0x5627a2b22b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.455+0000 7f476020c740 1 --2- 172.21.15.106:0/769010523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b28ae0 0x5627a2b39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.455+0000 7f476020c740 1 --2- 172.21.15.106:0/769010523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b22620 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.455+0000 7f476020c740 1 -- 172.21.15.106:0/769010523 >> 172.21.15.106:0/769010523 conn(0x5627a2abcd10 msgr2=0x5627a2b3df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.455+0000 7f476020c740 1 -- 172.21.15.106:0/769010523 shutdown_connections 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.455+0000 7f476020c740 1 -- 172.21.15.106:0/769010523 wait complete. 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.455+0000 7f476020c740 1 -- start start 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.455+0000 7f476020c740 1 Processor -- start 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f476020c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f476020c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 0x5627a2b35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f476020c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b2dc50 0x5627a2b38d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f476020c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5627a2b402c0 con 0x5627a2b280f0 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f476020c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5627a2b3fbf0 con 0x5627a2b28ae0 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f476020c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5627a2b40990 con 0x5627a2b2dc50 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f476020c740 10 client.0 _ll_register_callbacks cb 0x7f4740009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4756800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4755e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 0x5627a2b35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4756800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b357f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36368/0 (socket says 172.21.15.106:36368) 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4756800640 1 -- 172.21.15.106:0/2630150792 learned_addr learned my addr 172.21.15.106:0/2630150792 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4757200640 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b2dc50 0x5627a2b38d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4756800640 1 -- 172.21.15.106:0/2630150792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b2dc50 msgr2=0x5627a2b38d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4756800640 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b2dc50 0x5627a2b38d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4756800640 1 -- 172.21.15.106:0/2630150792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 msgr2=0x5627a2b35d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4756800640 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 0x5627a2b35d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4756800640 1 -- 172.21.15.106:0/2630150792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a2b39340 con 0x5627a2b280f0 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.456+0000 7f4755e00640 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 0x5627a2b35d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.457+0000 7f4756800640 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b357f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4748019580 tx=0x7f4748003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3615d344cd9e5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.457+0000 7f473be00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.457+0000 7f473be00640 1 -- 172.21.15.106:0/2630150792 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474803c070 con 0x5627a2b280f0 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.457+0000 7f476020c740 1 -- 172.21.15.106:0/2630150792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4748003aa0 con 0x5627a2b280f0 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.457+0000 7f473be00640 1 -- 172.21.15.106:0/2630150792 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f4748003800 con 0x5627a2b280f0 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.457+0000 7f473be00640 1 -- 172.21.15.106:0/2630150792 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f474804a050 con 0x5627a2b280f0 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.457+0000 7f476020c740 10 client.15093 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.457+0000 7f476020c740 1 -- 172.21.15.106:0/2630150792 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627a2badce0 con 0x5627a2b280f0 2024-04-28T12:51:05.457 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.457+0000 7f473be00640 1 -- 172.21.15.106:0/2630150792 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4748003aa0 con 0x5627a2b280f0 2024-04-28T12:51:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.458+0000 7f473be00640 1 -- 172.21.15.106:0/2630150792 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f47480043c0 con 0x5627a2b280f0 2024-04-28T12:51:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.458+0000 7f476020c740 10 client.15093 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.458+0000 7f476020c740 10 client.15093 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-28T12:51:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.458+0000 7f476020c740 20 client.15093 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.458+0000 7f476020c740 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x5627a2bb24f0 0x5627a2bb1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.458+0000 7f476020c740 4 client.15093 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.458+0000 7f476020c740 1 -- 172.21.15.106:0/2630150792 --> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] -- command(tid 0: {"prefix": "status"}) -- 0x5627a2a94080 con 0x5627a2bb24f0 2024-04-28T12:51:05.458 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.458+0000 7f4755e00640 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x5627a2bb24f0 0x5627a2bb1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f4755e00640 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x5627a2bb24f0 0x5627a2bb1200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79808abde62919cc server_cookie=c45fe0f5d3d80162 in_seq=0 out_seq=0 2024-04-28T12:51:05.459 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f473be00640 10 client.15093 ms_handle_connect on v2:172.21.15.106:6838/3863798001 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f473be00640 1 -- 172.21.15.106:0/2630150792 <== mds.2 v2:172.21.15.106:6838/3863798001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5627a2a94080 con 0x5627a2bb24f0 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f473be00640 10 client.15093 handle_command_reply: tid=0 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 client.15093 shutdown 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 -- 172.21.15.106:0/2630150792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 msgr2=0x5627a2b357f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b357f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4748019580 tx=0x7f4748003f00 comp rx=0 tx=0).stop 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 -- 172.21.15.106:0/2630150792 shutdown_connections 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x5627a2bb24f0 0x5627a2bb1200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5627a2b2dc50 0x5627a2b38d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5627a2b28ae0 0x5627a2b35d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f473be00640 0 client.15093 ms_handle_reset on v2:172.21.15.106:6838/3863798001 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 --2- 172.21.15.106:0/2630150792 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5627a2b280f0 0x5627a2b357f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 -- 172.21.15.106:0/2630150792 >> 172.21.15.106:0/2630150792 conn(0x5627a2abcd10 msgr2=0x5627a2b2d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 -- 172.21.15.106:0/2630150792 shutdown_connections 2024-04-28T12:51:05.460 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.459+0000 7f476020c740 1 -- 172.21.15.106:0/2630150792 wait complete. 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f476020c740 20 client.15093 trim_cache size 0 max 16384 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 2, 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24548, 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 98.476498746000004, 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:05.461 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:05.462 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:05.462 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 143.67565294600001 2024-04-28T12:51:05.462 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:05.462 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 -- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f473c0a3c40 msgr2=0x7f473c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f473c0a3c40 0x7f473c0a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5627a2b29340 tx=0x7f4740013040 comp rx=0 tx=0).stop 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 -- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4758106760 msgr2=0x7f47581a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4758106760 0x7f47581a2320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4750005e30 tx=0x7f4750003c00 comp rx=0 tx=0).stop 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 -- 172.21.15.106:0/860817508 shutdown_connections 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f473c0a3c40 0x7f473c0a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4758106760 0x7f47581a2320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4758105da0 0x7f475819b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 --2- 172.21.15.106:0/860817508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4758104bb0 0x7f475819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 -- 172.21.15.106:0/860817508 >> 172.21.15.106:0/860817508 conn(0x7f4758100120 msgr2=0x7f4758109170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.460+0000 7f475d400640 1 -- 172.21.15.106:0/860817508 shutdown_connections 2024-04-28T12:51:05.463 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.461+0000 7f475d400640 1 -- 172.21.15.106:0/860817508 wait complete. 2024-04-28T12:51:05.472 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:05 smithi106 ceph-mon[28081]: pgmap v387: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 127 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-04-28T12:51:05.472 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:05 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1448770066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:05.472 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:05 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3846376445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:05.475 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-28T12:51:05.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:05 smithi171 ceph-mon[63127]: pgmap v387: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 127 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-04-28T12:51:05.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:05 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1448770066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:05.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:05 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3846376445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:05.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:05 smithi129 ceph-mon[62418]: pgmap v387: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 127 B/s wr, 11 op/s; 14 B/s, 0 objects/s recovering 2024-04-28T12:51:05.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:05 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1448770066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:05.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:05 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3846376445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:05.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.664+0000 7f9113e00640 1 -- 172.21.15.106:0/3960020116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9114103960 msgr2=0x7f9114103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.664+0000 7f9113e00640 1 --2- 172.21.15.106:0/3960020116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9114103960 0x7f9114103de0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009900 tx=0x7f90fc02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.664+0000 7f9113e00640 1 -- 172.21.15.106:0/3960020116 shutdown_connections 2024-04-28T12:51:05.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.664+0000 7f9113e00640 1 --2- 172.21.15.106:0/3960020116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9114104320 0x7f911410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.664+0000 7f9113e00640 1 --2- 172.21.15.106:0/3960020116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9114103960 0x7f9114103de0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.664+0000 7f9113e00640 1 --2- 172.21.15.106:0/3960020116 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9114102770 0x7f9114102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.664+0000 7f9113e00640 1 -- 172.21.15.106:0/3960020116 >> 172.21.15.106:0/3960020116 conn(0x7f91140fdce0 msgr2=0x7f9114100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.664+0000 7f9113e00640 1 -- 172.21.15.106:0/3960020116 shutdown_connections 2024-04-28T12:51:05.663 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.664+0000 7f9113e00640 1 -- 172.21.15.106:0/3960020116 wait complete. 2024-04-28T12:51:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.666+0000 7f9113e00640 1 Processor -- start 2024-04-28T12:51:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.666+0000 7f9113e00640 1 -- start start 2024-04-28T12:51:05.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.666+0000 7f9113e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9114102770 0x7f911419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9112a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9114102770 0x7f911419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9112a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9114102770 0x7f911419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:47980/0 (socket says 172.21.15.106:47980) 2024-04-28T12:51:05.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9113e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9114103960 0x7f911419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9112a00640 1 -- 172.21.15.106:0/199586348 learned_addr learned my addr 172.21.15.106:0/199586348 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9113e00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9114104320 0x7f91141a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f911410e280 con 0x7f9114104320 2024-04-28T12:51:05.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f911410e100 con 0x7f9114103960 2024-04-28T12:51:05.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f911410e400 con 0x7f9114102770 2024-04-28T12:51:05.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f910a000640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9114103960 0x7f911419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9113400640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9114104320 0x7f91141a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9112a00640 1 -- 172.21.15.106:0/199586348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9114103960 msgr2=0x7f911419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9112a00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9114103960 0x7f911419b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9112a00640 1 -- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9114104320 msgr2=0x7f91141a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9112a00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9114104320 0x7f91141a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9112a00640 1 -- 172.21.15.106:0/199586348 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91141a2790 con 0x7f9114102770 2024-04-28T12:51:05.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.667+0000 7f9112a00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9114102770 0x7f911419ab70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f910400d770 tx=0x7f910400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.668+0000 7f910be00640 1 -- 172.21.15.106:0/199586348 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104017070 con 0x7f9114102770 2024-04-28T12:51:05.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.668+0000 7f910be00640 1 -- 172.21.15.106:0/199586348 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9104009c30 con 0x7f9114102770 2024-04-28T12:51:05.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.668+0000 7f910be00640 1 -- 172.21.15.106:0/199586348 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104003cc0 con 0x7f9114102770 2024-04-28T12:51:05.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.668+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91141a2a80 con 0x7f9114102770 2024-04-28T12:51:05.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.668+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91141a2f80 con 0x7f9114102770 2024-04-28T12:51:05.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.671+0000 7f911c3fd740 1 Processor -- start 2024-04-28T12:51:05.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.671+0000 7f911c3fd740 1 -- start start 2024-04-28T12:51:05.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff807e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff807ecb0 0x559ff80837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff8083e20 0x559ff80862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f911c3fd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559ff7fe2600 con 0x559ff8083e20 2024-04-28T12:51:05.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f911c3fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ff7f7f100 con 0x559ff807e2c0 2024-04-28T12:51:05.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f911c3fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559ff7fde5c0 con 0x559ff807ecb0 2024-04-28T12:51:05.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f9113400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff807e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f9113400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff807e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:33134/0 (socket says 172.21.15.106:33134) 2024-04-28T12:51:05.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f9113400640 1 -- 172.21.15.106:0/3216701248 learned_addr learned my addr 172.21.15.106:0/3216701248 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f910a000640 1 --2- 172.21.15.106:0/3216701248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff8083e20 0x559ff80862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f9113400640 1 -- 172.21.15.106:0/3216701248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff807ecb0 msgr2=0x559ff80837b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-28T12:51:05.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f9113400640 1 --2- 172.21.15.106:0/3216701248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff807ecb0 0x559ff80837b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f9113400640 1 -- 172.21.15.106:0/3216701248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff8083e20 msgr2=0x559ff80862b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f9113400640 1 --2- 172.21.15.106:0/3216701248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff8083e20 0x559ff80862b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.672+0000 7f9113400640 1 -- 172.21.15.106:0/3216701248 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ff7cf42d0 con 0x559ff807e2c0 2024-04-28T12:51:05.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.673+0000 7f9113400640 1 --2- 172.21.15.106:0/3216701248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff807e6e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f910c009fc0 tx=0x7f910c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804380d11e707e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.673+0000 7f9103e00640 1 -- 172.21.15.106:0/3216701248 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c003b70 con 0x559ff807e2c0 2024-04-28T12:51:05.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.673+0000 7f9103e00640 1 -- 172.21.15.106:0/3216701248 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f910c003d10 con 0x559ff807e2c0 2024-04-28T12:51:05.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.673+0000 7f9103e00640 1 -- 172.21.15.106:0/3216701248 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c011a60 con 0x559ff807e2c0 2024-04-28T12:51:05.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.673+0000 7f910be00640 1 -- 172.21.15.106:0/199586348 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9104003e60 con 0x7f9114102770 2024-04-28T12:51:05.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 -- 172.21.15.106:0/3216701248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 msgr2=0x559ff807e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3216701248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff807e6e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f910c009fc0 tx=0x7f910c010860 comp rx=0 tx=0).stop 2024-04-28T12:51:05.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 -- 172.21.15.106:0/3216701248 shutdown_connections 2024-04-28T12:51:05.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3216701248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff8083e20 0x559ff80862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3216701248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff807ecb0 0x559ff80837b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3216701248 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff807e6e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 -- 172.21.15.106:0/3216701248 >> 172.21.15.106:0/3216701248 conn(0x559ff8012d10 msgr2=0x559ff8079d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 -- 172.21.15.106:0/3216701248 shutdown_connections 2024-04-28T12:51:05.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 -- 172.21.15.106:0/3216701248 wait complete. 2024-04-28T12:51:05.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.674+0000 7f911c3fd740 1 -- start start 2024-04-28T12:51:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.675+0000 7f911c3fd740 1 Processor -- start 2024-04-28T12:51:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.675+0000 7f910be00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90f80a3bd0 0x7f90f80a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.675+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff807e2c0 0x559ff8088740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.675+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff8088c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.675+0000 7f910be00640 1 -- 172.21.15.106:0/199586348 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f91040d0320 con 0x7f9114102770 2024-04-28T12:51:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.675+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff8083e20 0x559ff808fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.675+0000 7f911c3fd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559ff8086dd0 con 0x559ff807ecb0 2024-04-28T12:51:05.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.675+0000 7f911c3fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ff7f69f70 con 0x559ff8083e20 2024-04-28T12:51:05.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.675+0000 7f911c3fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559ff8086f50 con 0x559ff807e2c0 2024-04-28T12:51:05.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f911c3fd740 10 client.0 _ll_register_callbacks cb 0x559ff7fd8480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:05.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f9113400640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90f80a3bd0 0x7f90f80a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f910a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff8088c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f910a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff8088c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36404/0 (socket says 172.21.15.106:36404) 2024-04-28T12:51:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f910a000640 1 -- 172.21.15.106:0/4032510853 learned_addr learned my addr 172.21.15.106:0/4032510853 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f9112a00640 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff807e2c0 0x559ff8088740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f9113400640 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff8083e20 0x559ff808fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f910a000640 1 -- 172.21.15.106:0/4032510853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff807e2c0 msgr2=0x559ff8088740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f910a000640 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff807e2c0 0x559ff8088740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f910a000640 1 -- 172.21.15.106:0/4032510853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff8083e20 msgr2=0x559ff808fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f910a000640 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff8083e20 0x559ff808fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.676+0000 7f910a000640 1 -- 172.21.15.106:0/4032510853 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ff7dbe6b0 con 0x559ff807ecb0 2024-04-28T12:51:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.677+0000 7f910a000640 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff8088c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005ab0 tx=0x7f90fc0317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be98bd0741076d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.677+0000 7f9113400640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90f80a3bd0 0x7f90f80a6090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f910c009dd0 tx=0x7f910c010fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.677+0000 7f9100c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.677+0000 7f9100c00640 1 -- 172.21.15.106:0/4032510853 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc035040 con 0x559ff807ecb0 2024-04-28T12:51:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.677+0000 7f9100c00640 1 -- 172.21.15.106:0/4032510853 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f90fc004240 con 0x559ff807ecb0 2024-04-28T12:51:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.677+0000 7f911c3fd740 1 -- 172.21.15.106:0/4032510853 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910c003d10 con 0x559ff807ecb0 2024-04-28T12:51:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.677+0000 7f9100c00640 1 -- 172.21.15.106:0/4032510853 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90fc04e050 con 0x559ff807ecb0 2024-04-28T12:51:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.677+0000 7f911c3fd740 10 client.15105 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.677+0000 7f911c3fd740 1 -- 172.21.15.106:0/4032510853 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ff8092d10 con 0x559ff807ecb0 2024-04-28T12:51:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.678+0000 7f9100c00640 1 -- 172.21.15.106:0/4032510853 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f910c003d10 con 0x559ff807ecb0 2024-04-28T12:51:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.678+0000 7f9100c00640 1 -- 172.21.15.106:0/4032510853 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f90fc004ad0 con 0x559ff807ecb0 2024-04-28T12:51:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.678+0000 7f911c3fd740 10 client.15105 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.678+0000 7f911c3fd740 10 client.15105 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-28T12:51:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.678+0000 7f911c3fd740 20 client.15105 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.678+0000 7f911c3fd740 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x559ff8095330 0x559ff80967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.678+0000 7f911c3fd740 4 client.15105 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.678+0000 7f911c3fd740 1 -- 172.21.15.106:0/4032510853 --> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ff7ff0210 con 0x559ff8095330 2024-04-28T12:51:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.678+0000 7f9112a00640 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x559ff8095330 0x559ff80967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.679+0000 7f9112a00640 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x559ff8095330 0x559ff80967a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb995eb835d09874 server_cookie=5f5ca194f7775c7c in_seq=0 out_seq=0 2024-04-28T12:51:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.679+0000 7f9100c00640 10 client.15105 ms_handle_connect on v2:172.21.15.106:6840/92504617 2024-04-28T12:51:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f9100c00640 1 -- 172.21.15.106:0/4032510853 <== mds.1 v2:172.21.15.106:6840/92504617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ff7ff0210 con 0x559ff8095330 2024-04-28T12:51:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f9100c00640 10 client.15105 handle_command_reply: tid=0 2024-04-28T12:51:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 client.15105 shutdown 2024-04-28T12:51:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 -- 172.21.15.106:0/4032510853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 msgr2=0x559ff8088c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff8088c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005ab0 tx=0x7f90fc0317d0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 -- 172.21.15.106:0/4032510853 shutdown_connections 2024-04-28T12:51:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x559ff8095330 0x559ff80967a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff8088c80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff8083e20 0x559ff808fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f9100c00640 0 client.15105 ms_handle_reset on v2:172.21.15.106:6840/92504617 2024-04-28T12:51:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 --2- 172.21.15.106:0/4032510853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff807e2c0 0x559ff8088740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 -- 172.21.15.106:0/4032510853 >> 172.21.15.106:0/4032510853 conn(0x559ff8012d10 msgr2=0x559ff8078f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 -- 172.21.15.106:0/4032510853 shutdown_connections 2024-04-28T12:51:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 1 -- 172.21.15.106:0/4032510853 wait complete. 2024-04-28T12:51:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.680+0000 7f911c3fd740 20 client.15105 trim_cache size 0 max 16384 2024-04-28T12:51:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.700+0000 7f911c3fd740 1 Processor -- start 2024-04-28T12:51:05.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.700+0000 7f911c3fd740 1 -- start start 2024-04-28T12:51:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807e2c0 0x559ff8088350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807ecb0 0x559ff808fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff8088890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f911c3fd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559ff8094ae0 con 0x559ff807e2c0 2024-04-28T12:51:05.711 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f911c3fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ff80951b0 con 0x559ff807ecb0 2024-04-28T12:51:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f911c3fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559ff8095880 con 0x559ff8083e20 2024-04-28T12:51:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f9112a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807e2c0 0x559ff8088350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f910a000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807ecb0 0x559ff808fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f9113400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff8088890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.712 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f910a000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807ecb0 0x559ff808fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:33156/0 (socket says 172.21.15.106:33156) 2024-04-28T12:51:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f910a000640 1 -- 172.21.15.106:0/2247977332 learned_addr learned my addr 172.21.15.106:0/2247977332 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f9112a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807e2c0 0x559ff8088350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36416/0 (socket says 172.21.15.106:36416) 2024-04-28T12:51:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.701+0000 7f910a000640 1 -- 172.21.15.106:0/2247977332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 msgr2=0x559ff8088890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f910a000640 1 --2- 172.21.15.106:0/2247977332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff8088890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.713 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f910a000640 1 -- 172.21.15.106:0/2247977332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807e2c0 msgr2=0x559ff8088350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f910a000640 1 --2- 172.21.15.106:0/2247977332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807e2c0 0x559ff8088350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f910a000640 1 -- 172.21.15.106:0/2247977332 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ff807f090 con 0x559ff807ecb0 2024-04-28T12:51:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f9113400640 1 --2- 172.21.15.106:0/2247977332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff8088890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f9112a00640 1 --2- 172.21.15.106:0/2247977332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807e2c0 0x559ff8088350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:05.714 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f910a000640 1 --2- 172.21.15.106:0/2247977332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807ecb0 0x559ff808fc80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90fc000c00 tx=0x7f90fc005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85225a491023c835 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f9103400640 1 -- 172.21.15.106:0/2247977332 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc035030 con 0x559ff807ecb0 2024-04-28T12:51:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f9103400640 1 -- 172.21.15.106:0/2247977332 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90fc005450 con 0x559ff807ecb0 2024-04-28T12:51:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.702+0000 7f9103400640 1 -- 172.21.15.106:0/2247977332 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc004cc0 con 0x559ff807ecb0 2024-04-28T12:51:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 -- 172.21.15.106:0/2247977332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807ecb0 msgr2=0x559ff808fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 --2- 172.21.15.106:0/2247977332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807ecb0 0x559ff808fc80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90fc000c00 tx=0x7f90fc005990 comp rx=0 tx=0).stop 2024-04-28T12:51:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 -- 172.21.15.106:0/2247977332 shutdown_connections 2024-04-28T12:51:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 --2- 172.21.15.106:0/2247977332 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff8088890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 --2- 172.21.15.106:0/2247977332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807ecb0 0x559ff808fc80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 --2- 172.21.15.106:0/2247977332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807e2c0 0x559ff8088350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 -- 172.21.15.106:0/2247977332 >> 172.21.15.106:0/2247977332 conn(0x559ff8012d10 msgr2=0x559ff8080110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 -- 172.21.15.106:0/2247977332 shutdown_connections 2024-04-28T12:51:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 -- 172.21.15.106:0/2247977332 wait complete. 2024-04-28T12:51:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.703+0000 7f911c3fd740 1 -- start start 2024-04-28T12:51:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f911c3fd740 1 Processor -- start 2024-04-28T12:51:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff808bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff808c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.717 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f911c3fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff808f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f911c3fd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559ff8096620 con 0x559ff807ecb0 2024-04-28T12:51:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f911c3fd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ff8095f50 con 0x559ff807e2c0 2024-04-28T12:51:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f911c3fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559ff8096cf0 con 0x559ff8083e20 2024-04-28T12:51:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f911c3fd740 10 client.0 _ll_register_callbacks cb 0x7f90fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f910a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff808c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f910a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff808c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36428/0 (socket says 172.21.15.106:36428) 2024-04-28T12:51:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f910a000640 1 -- 172.21.15.106:0/3649346204 learned_addr learned my addr 172.21.15.106:0/3649346204 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f9112a00640 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff808bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.704+0000 7f9113400640 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff808f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f910a000640 1 -- 172.21.15.106:0/3649346204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 msgr2=0x559ff808f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f910a000640 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff808f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f910a000640 1 -- 172.21.15.106:0/3649346204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 msgr2=0x559ff808bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f910a000640 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff808bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f910a000640 1 -- 172.21.15.106:0/3649346204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ff808f660 con 0x559ff807ecb0 2024-04-28T12:51:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f9113400640 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff808f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f910a000640 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff808c050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005a60 tx=0x7f90fc004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b0e36bb185d37a server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f9100c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:05.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f9100c00640 1 -- 172.21.15.106:0/3649346204 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc035070 con 0x559ff807ecb0 2024-04-28T12:51:05.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f9100c00640 1 -- 172.21.15.106:0/3649346204 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f90fc005160 con 0x559ff807ecb0 2024-04-28T12:51:05.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.705+0000 7f911c3fd740 1 -- 172.21.15.106:0/3649346204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90fc005450 con 0x559ff807ecb0 2024-04-28T12:51:05.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f9100c00640 1 -- 172.21.15.106:0/3649346204 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90fc069050 con 0x559ff807ecb0 2024-04-28T12:51:05.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f911c3fd740 10 client.15111 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:05.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f911c3fd740 1 -- 172.21.15.106:0/3649346204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ff80fba80 con 0x559ff807ecb0 2024-04-28T12:51:05.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f9100c00640 1 -- 172.21.15.106:0/3649346204 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90fc005450 con 0x559ff807ecb0 2024-04-28T12:51:05.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f9100c00640 1 -- 172.21.15.106:0/3649346204 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f90fc064030 con 0x559ff807ecb0 2024-04-28T12:51:05.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f911c3fd740 10 client.15111 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:05.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f911c3fd740 10 client.15111 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-28T12:51:05.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f911c3fd740 20 client.15111 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:05.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x559ff81004f0 0x559ff80ff200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f911c3fd740 4 client.15111 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:05.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.706+0000 7f911c3fd740 1 -- 172.21.15.106:0/3649346204 --> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] -- command(tid 0: {"prefix": "status"}) -- 0x559ff7fea080 con 0x559ff81004f0 2024-04-28T12:51:05.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.707+0000 7f9112a00640 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x559ff81004f0 0x559ff80ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.707+0000 7f9112a00640 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x559ff81004f0 0x559ff80ff200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73f5002e63eb9821 server_cookie=7d15bd624833ee86 in_seq=0 out_seq=0 2024-04-28T12:51:05.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.707+0000 7f9100c00640 10 client.15111 ms_handle_connect on v2:172.21.15.106:6840/92504617 2024-04-28T12:51:05.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.707+0000 7f9100c00640 1 -- 172.21.15.106:0/3649346204 <== mds.1 v2:172.21.15.106:6840/92504617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559ff7fea080 con 0x559ff81004f0 2024-04-28T12:51:05.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.707+0000 7f9100c00640 10 client.15111 handle_command_reply: tid=0 2024-04-28T12:51:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.707+0000 7f911c3fd740 1 client.15111 shutdown 2024-04-28T12:51:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 -- 172.21.15.106:0/3649346204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 msgr2=0x559ff808c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff808c050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f90fc005a60 tx=0x7f90fc004cf0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 -- 172.21.15.106:0/3649346204 shutdown_connections 2024-04-28T12:51:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x559ff81004f0 0x559ff80ff200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559ff807ecb0 0x559ff808c050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559ff8083e20 0x559ff808f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f9100c00640 0 client.15111 ms_handle_reset on v2:172.21.15.106:6840/92504617 2024-04-28T12:51:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 --2- 172.21.15.106:0/3649346204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ff807e2c0 0x559ff808bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 -- 172.21.15.106:0/3649346204 >> 172.21.15.106:0/3649346204 conn(0x559ff8012d10 msgr2=0x559ff8088330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 -- 172.21.15.106:0/3649346204 shutdown_connections 2024-04-28T12:51:05.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 1 -- 172.21.15.106:0/3649346204 wait complete. 2024-04-28T12:51:05.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.708+0000 7f911c3fd740 20 client.15111 trim_cache size 0 max 16384 2024-04-28T12:51:05.726 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:05.726 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 1, 2024-04-28T12:51:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24566, 2024-04-28T12:51:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 99.744781477000004, 2024-04-28T12:51:05.727 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:05.728 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 139.81724287700001 2024-04-28T12:51:05.728 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90f80a3bd0 msgr2=0x7f90f80a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90f80a3bd0 0x7f90f80a6090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f910c009dd0 tx=0x7f910c010fd0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9114102770 msgr2=0x7f911419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9114102770 0x7f911419ab70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f910400d770 tx=0x7f910400dc30 comp rx=0 tx=0).stop 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 shutdown_connections 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f90f80a3bd0 0x7f90f80a6090 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9114104320 0x7f91141a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9114103960 0x7f911419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 --2- 172.21.15.106:0/199586348 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9114102770 0x7f911419ab70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 >> 172.21.15.106:0/199586348 conn(0x7f91140fdce0 msgr2=0x7f9114074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 shutdown_connections 2024-04-28T12:51:05.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.709+0000 7f9113e00640 1 -- 172.21.15.106:0/199586348 wait complete. 2024-04-28T12:51:05.731 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-28T12:51:05.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.933+0000 7f42b9200640 1 -- 172.21.15.106:0/2085806255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4102770 msgr2=0x7f42b4102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.933+0000 7f42b9200640 1 --2- 172.21.15.106:0/2085806255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4102770 0x7f42b4102b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f42a4009900 tx=0x7f42a402a3f0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.933+0000 7f42b9200640 1 -- 172.21.15.106:0/2085806255 shutdown_connections 2024-04-28T12:51:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.933+0000 7f42b9200640 1 --2- 172.21.15.106:0/2085806255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4104320 0x7f42b410abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.933+0000 7f42b9200640 1 --2- 172.21.15.106:0/2085806255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4103960 0x7f42b4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.933+0000 7f42b9200640 1 --2- 172.21.15.106:0/2085806255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4102770 0x7f42b4102b90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.933+0000 7f42b9200640 1 -- 172.21.15.106:0/2085806255 >> 172.21.15.106:0/2085806255 conn(0x7f42b40fdce0 msgr2=0x7f42b4100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.933+0000 7f42b9200640 1 -- 172.21.15.106:0/2085806255 shutdown_connections 2024-04-28T12:51:05.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.933+0000 7f42b9200640 1 -- 172.21.15.106:0/2085806255 wait complete. 2024-04-28T12:51:05.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.935+0000 7f42b9200640 1 Processor -- start 2024-04-28T12:51:05.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.935+0000 7f42b9200640 1 -- start start 2024-04-28T12:51:05.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.935+0000 7f42b9200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4102770 0x7f42b419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.935+0000 7f42b9200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4103960 0x7f42b419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.935+0000 7f42b9200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4104320 0x7f42b41a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.935+0000 7f42b9200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f42b410e2c0 con 0x7f42b4104320 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.935+0000 7f42b9200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f42b410e140 con 0x7f42b4103960 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.935+0000 7f42b9200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f42b410e440 con 0x7f42b4102770 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4104320 0x7f42b41a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4102770 0x7f42b419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4104320 0x7f42b41a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36452/0 (socket says 172.21.15.106:36452) 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3e00640 1 -- 172.21.15.106:0/2550862906 learned_addr learned my addr 172.21.15.106:0/2550862906 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42abe00640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4103960 0x7f42b419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3e00640 1 -- 172.21.15.106:0/2550862906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4102770 msgr2=0x7f42b419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3e00640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4102770 0x7f42b419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3e00640 1 -- 172.21.15.106:0/2550862906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4103960 msgr2=0x7f42b419b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.934 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3e00640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4103960 0x7f42b419b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3e00640 1 -- 172.21.15.106:0/2550862906 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b41a27d0 con 0x7f42b4104320 2024-04-28T12:51:05.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.936+0000 7f42b3e00640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4104320 0x7f42b41a2160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00eb10 tx=0x7f42ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.937+0000 7f42b0c00640 1 -- 172.21.15.106:0/2550862906 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac004b20 con 0x7f42b4104320 2024-04-28T12:51:05.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.937+0000 7f42b0c00640 1 -- 172.21.15.106:0/2550862906 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac004cc0 con 0x7f42b4104320 2024-04-28T12:51:05.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.937+0000 7f42b9200640 1 -- 172.21.15.106:0/2550862906 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b41a2ac0 con 0x7f42b4104320 2024-04-28T12:51:05.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.937+0000 7f42b0c00640 1 -- 172.21.15.106:0/2550862906 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac010670 con 0x7f42b4104320 2024-04-28T12:51:05.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.937+0000 7f42b9200640 1 -- 172.21.15.106:0/2550862906 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b41a2fc0 con 0x7f42b4104320 2024-04-28T12:51:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.940+0000 7f42b0c00640 1 -- 172.21.15.106:0/2550862906 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f42ac005020 con 0x7f42b4104320 2024-04-28T12:51:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.940+0000 7f42bd78e740 1 Processor -- start 2024-04-28T12:51:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.940+0000 7f42bd78e740 1 -- start start 2024-04-28T12:51:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.940+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa32cb0 0x5616efa377b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 0x5616efa3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.937 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42bd78e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5616ef996600 con 0x5616efa322c0 2024-04-28T12:51:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42bd78e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616ef933100 con 0x5616efa32cb0 2024-04-28T12:51:05.938 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42bd78e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5616ef9925c0 con 0x5616efa37e20 2024-04-28T12:51:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42b3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa32cb0 0x5616efa377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42b3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa32cb0 0x5616efa377b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:33204/0 (socket says 172.21.15.106:33204) 2024-04-28T12:51:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42b3400640 1 -- 172.21.15.106:0/2744311468 learned_addr learned my addr 172.21.15.106:0/2744311468 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42abe00640 1 --2- 172.21.15.106:0/2744311468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42b3400640 1 -- 172.21.15.106:0/2744311468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 msgr2=0x5616efa3a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42b3400640 1 --2- 172.21.15.106:0/2744311468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 0x5616efa3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42b3400640 1 -- 172.21.15.106:0/2744311468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 msgr2=0x5616efa326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42b3400640 1 --2- 172.21.15.106:0/2744311468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa326e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.940 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.941+0000 7f42b3400640 1 -- 172.21.15.106:0/2744311468 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616ef6a82d0 con 0x5616efa32cb0 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.942+0000 7f42b3400640 1 --2- 172.21.15.106:0/2744311468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa32cb0 0x5616efa377b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f42a4006010 tx=0x7f42a4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450f3cac4274db27 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.942+0000 7f42b0c00640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3c40 0x7f42980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.942+0000 7f42b0c00640 1 -- 172.21.15.106:0/2550862906 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f42ac0d0f50 con 0x7f42b4104320 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.942+0000 7f42abe00640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3c40 0x7f42980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.942+0000 7f42a8c00640 1 -- 172.21.15.106:0/2744311468 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4035030 con 0x5616efa32cb0 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.942+0000 7f42a8c00640 1 -- 172.21.15.106:0/2744311468 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a4040070 con 0x5616efa32cb0 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.942+0000 7f42a8c00640 1 -- 172.21.15.106:0/2744311468 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4031890 con 0x5616efa32cb0 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42bd78e740 1 -- 172.21.15.106:0/2744311468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa32cb0 msgr2=0x5616efa377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42abe00640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3c40 0x7f42980a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5616efa33510 tx=0x7f429c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2744311468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa32cb0 0x5616efa377b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f42a4006010 tx=0x7f42a4004040 comp rx=0 tx=0).stop 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42bd78e740 1 -- 172.21.15.106:0/2744311468 shutdown_connections 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2744311468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 0x5616efa3a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2744311468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa32cb0 0x5616efa377b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.941 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2744311468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42bd78e740 1 -- 172.21.15.106:0/2744311468 >> 172.21.15.106:0/2744311468 conn(0x5616ef9c6d10 msgr2=0x5616efa2dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42bd78e740 1 -- 172.21.15.106:0/2744311468 shutdown_connections 2024-04-28T12:51:05.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.943+0000 7f42bd78e740 1 -- 172.21.15.106:0/2744311468 wait complete. 2024-04-28T12:51:05.942 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42bd78e740 1 -- start start 2024-04-28T12:51:05.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42bd78e740 1 Processor -- start 2024-04-28T12:51:05.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa322c0 0x5616efa3c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa32cb0 0x5616efa3cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 0x5616efa43c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42bd78e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5616efa3ad70 con 0x5616efa32cb0 2024-04-28T12:51:05.945 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42bd78e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616ef91df70 con 0x5616efa322c0 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42bd78e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5616efa3aef0 con 0x5616efa37e20 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42bd78e740 10 client.0 _ll_register_callbacks cb 0x5616ef98c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.944+0000 7f42abe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa32cb0 0x5616efa3cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa322c0 0x5616efa3c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 0x5616efa43c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa322c0 0x5616efa3c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:33214/0 (socket says 172.21.15.106:33214) 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42abe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa32cb0 0x5616efa3cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36460/0 (socket says 172.21.15.106:36460) 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3400640 1 -- 172.21.15.106:0/3682609439 learned_addr learned my addr 172.21.15.106:0/3682609439 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3400640 1 -- 172.21.15.106:0/3682609439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 msgr2=0x5616efa43c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3400640 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 0x5616efa43c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3400640 1 -- 172.21.15.106:0/3682609439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa32cb0 msgr2=0x5616efa3cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3400640 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa32cb0 0x5616efa3cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.946 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3400640 1 -- 172.21.15.106:0/3682609439 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616ef7726b0 con 0x5616efa322c0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3e00640 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 0x5616efa43c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42abe00640 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa32cb0 0x5616efa3cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42b3400640 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa322c0 0x5616efa3c690 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f42a402a910 tx=0x7f42a4004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0c69befbb386ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.945+0000 7f42a1600640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.946+0000 7f42a1600640 1 -- 172.21.15.106:0/3682609439 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4053060 con 0x5616efa322c0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.946+0000 7f42bd78e740 1 -- 172.21.15.106:0/3682609439 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42a4040070 con 0x5616efa322c0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.946+0000 7f42a1600640 1 -- 172.21.15.106:0/3682609439 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f42a4003a00 con 0x5616efa322c0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.946+0000 7f42a1600640 1 -- 172.21.15.106:0/3682609439 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42a405a050 con 0x5616efa322c0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.946+0000 7f42bd78e740 10 client.25226 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.946+0000 7f42bd78e740 1 -- 172.21.15.106:0/3682609439 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5616efa46bf0 con 0x5616efa322c0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.946+0000 7f42a1600640 1 -- 172.21.15.106:0/3682609439 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a4040070 con 0x5616efa322c0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.947+0000 7f42a1600640 1 -- 172.21.15.106:0/3682609439 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f42a4045600 con 0x5616efa322c0 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.947+0000 7f42bd78e740 10 client.25226 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:05.947 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.947+0000 7f42bd78e740 10 client.25226 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:51:05.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.947+0000 7f42bd78e740 20 client.25226 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:05.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.947+0000 7f42bd78e740 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5616efa48dd0 0x5616efa4a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.947+0000 7f42bd78e740 4 client.25226 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:05.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.947+0000 7f42bd78e740 1 -- 172.21.15.106:0/3682609439 --> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616ef9a4210 con 0x5616efa48dd0 2024-04-28T12:51:05.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.947+0000 7f42abe00640 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5616efa48dd0 0x5616efa4a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.948+0000 7f42abe00640 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5616efa48dd0 0x5616efa4a240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275f0aa183ea62de server_cookie=e55e31d780bf6c34 in_seq=0 out_seq=0 2024-04-28T12:51:05.948 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.948+0000 7f42a1600640 10 client.25226 ms_handle_connect on v2:172.21.15.129:6838/4208792665 2024-04-28T12:51:05.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42a1600640 1 -- 172.21.15.106:0/3682609439 <== mds.0 v2:172.21.15.129:6838/4208792665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616ef9a4210 con 0x5616efa48dd0 2024-04-28T12:51:05.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42a1600640 10 client.25226 handle_command_reply: tid=0 2024-04-28T12:51:05.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 client.25226 shutdown 2024-04-28T12:51:05.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 -- 172.21.15.106:0/3682609439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa322c0 msgr2=0x5616efa3c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.949 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa322c0 0x5616efa3c690 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f42a402a910 tx=0x7f42a4004460 comp rx=0 tx=0).stop 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 -- 172.21.15.106:0/3682609439 shutdown_connections 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5616efa48dd0 0x5616efa4a240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa32cb0 0x5616efa3cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42a1600640 0 client.25226 ms_handle_reset on v2:172.21.15.129:6838/4208792665 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa37e20 0x5616efa43c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 --2- 172.21.15.106:0/3682609439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa322c0 0x5616efa3c690 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 -- 172.21.15.106:0/3682609439 >> 172.21.15.106:0/3682609439 conn(0x5616ef9c6d10 msgr2=0x5616efa2cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.949+0000 7f42bd78e740 1 -- 172.21.15.106:0/3682609439 shutdown_connections 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.950+0000 7f42bd78e740 1 -- 172.21.15.106:0/3682609439 wait complete. 2024-04-28T12:51:05.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.950+0000 7f42bd78e740 20 client.25226 trim_cache size 0 max 16384 2024-04-28T12:51:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.969+0000 7f42bd78e740 1 Processor -- start 2024-04-28T12:51:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.969+0000 7f42bd78e740 1 -- start start 2024-04-28T12:51:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.969+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa30c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.969+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 0x5616efa43bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.969+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa31140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.969+0000 7f42bd78e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5616efa48580 con 0x5616efa322c0 2024-04-28T12:51:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.969+0000 7f42bd78e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616efa48c50 con 0x5616efa37e20 2024-04-28T12:51:05.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.969+0000 7f42bd78e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5616efa49320 con 0x5616efa32cb0 2024-04-28T12:51:05.967 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa31140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa31140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:33216/0 (socket says 172.21.15.106:33216) 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3e00640 1 -- 172.21.15.106:0/4240836692 learned_addr learned my addr 172.21.15.106:0/4240836692 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42abe00640 1 --2- 172.21.15.106:0/4240836692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 0x5616efa43bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3400640 1 --2- 172.21.15.106:0/4240836692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3e00640 1 -- 172.21.15.106:0/4240836692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 msgr2=0x5616efa43bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3e00640 1 --2- 172.21.15.106:0/4240836692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 0x5616efa43bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3e00640 1 -- 172.21.15.106:0/4240836692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 msgr2=0x5616efa30c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3e00640 1 --2- 172.21.15.106:0/4240836692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa30c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3e00640 1 -- 172.21.15.106:0/4240836692 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616efa29680 con 0x5616efa37e20 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.970+0000 7f42b3e00640 1 --2- 172.21.15.106:0/4240836692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa31140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f42ac099120 tx=0x7f42ac09c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae79f11555a070e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.971+0000 7f42a3e00640 1 -- 172.21.15.106:0/4240836692 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0bb070 con 0x5616efa37e20 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.971+0000 7f42a3e00640 1 -- 172.21.15.106:0/4240836692 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac09ccc0 con 0x5616efa37e20 2024-04-28T12:51:05.968 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.971+0000 7f42a3e00640 1 -- 172.21.15.106:0/4240836692 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0b5030 con 0x5616efa37e20 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.971+0000 7f42bd78e740 1 -- 172.21.15.106:0/4240836692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 msgr2=0x5616efa31140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.971+0000 7f42bd78e740 1 --2- 172.21.15.106:0/4240836692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa31140 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f42ac099120 tx=0x7f42ac09c8c0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.972+0000 7f42bd78e740 1 -- 172.21.15.106:0/4240836692 shutdown_connections 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.972+0000 7f42bd78e740 1 --2- 172.21.15.106:0/4240836692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa31140 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.972+0000 7f42bd78e740 1 --2- 172.21.15.106:0/4240836692 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 0x5616efa43bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.972+0000 7f42bd78e740 1 --2- 172.21.15.106:0/4240836692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa30c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.972+0000 7f42bd78e740 1 -- 172.21.15.106:0/4240836692 >> 172.21.15.106:0/4240836692 conn(0x5616ef9c6d10 msgr2=0x5616efa28df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.972+0000 7f42bd78e740 1 -- 172.21.15.106:0/4240836692 shutdown_connections 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.972+0000 7f42bd78e740 1 -- 172.21.15.106:0/4240836692 wait complete. 2024-04-28T12:51:05.973 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.972+0000 7f42bd78e740 1 -- start start 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.972+0000 7f42bd78e740 1 Processor -- start 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa3f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 0x5616efa3fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42bd78e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa42dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42bd78e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5616efa4a0c0 con 0x5616efa322c0 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42bd78e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616efa499f0 con 0x5616efa37e20 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42bd78e740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5616efa4a790 con 0x5616efa32cb0 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42bd78e740 10 client.0 _ll_register_callbacks cb 0x7f429c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa42dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa42dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:33228/0 (socket says 172.21.15.106:33228) 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3e00640 1 -- 172.21.15.106:0/2126731046 learned_addr learned my addr 172.21.15.106:0/2126731046 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3400640 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa3f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42abe00640 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 0x5616efa3fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3e00640 1 -- 172.21.15.106:0/2126731046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 msgr2=0x5616efa3fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3e00640 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 0x5616efa3fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3e00640 1 -- 172.21.15.106:0/2126731046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 msgr2=0x5616efa3f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3e00640 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa3f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3e00640 1 -- 172.21.15.106:0/2126731046 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616efa433c0 con 0x5616efa37e20 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42abe00640 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 0x5616efa3fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.973+0000 7f42b3400640 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa3f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.974+0000 7f42b3e00640 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa42dd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42ac099690 tx=0x7f42ac00efd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d611795ed245c36 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.974+0000 7f42a1600640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.974+0000 7f42a1600640 1 -- 172.21.15.106:0/2126731046 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac080060 con 0x5616efa37e20 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.974+0000 7f42a1600640 1 -- 172.21.15.106:0/2126731046 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f42ac0bd050 con 0x5616efa37e20 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.974+0000 7f42bd78e740 1 -- 172.21.15.106:0/2126731046 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ac09ccc0 con 0x5616efa37e20 2024-04-28T12:51:05.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.974+0000 7f42a1600640 1 -- 172.21.15.106:0/2126731046 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42ac0b5070 con 0x5616efa37e20 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.974+0000 7f42bd78e740 10 client.25232 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.974+0000 7f42bd78e740 1 -- 172.21.15.106:0/2126731046 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5616efaaf310 con 0x5616efa37e20 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.974+0000 7f42a1600640 1 -- 172.21.15.106:0/2126731046 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac09ccc0 con 0x5616efa37e20 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.975+0000 7f42a1600640 1 -- 172.21.15.106:0/2126731046 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f42ac0a5b80 con 0x5616efa37e20 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.975+0000 7f42bd78e740 10 client.25232 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.975+0000 7f42bd78e740 10 client.25232 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.975+0000 7f42bd78e740 20 client.25232 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.975+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5616efab3990 0x5616efab04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.975+0000 7f42bd78e740 4 client.25232 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.975+0000 7f42bd78e740 1 -- 172.21.15.106:0/2126731046 --> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] -- command(tid 0: {"prefix": "status"}) -- 0x5616ef99e080 con 0x5616efab3990 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.975+0000 7f42b3400640 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5616efab3990 0x5616efab04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42b3400640 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5616efab3990 0x5616efab04a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c835d7c16d02acd server_cookie=a8a3881d6def2b33 in_seq=0 out_seq=0 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42a1600640 10 client.25232 ms_handle_connect on v2:172.21.15.129:6838/4208792665 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42a1600640 1 -- 172.21.15.106:0/2126731046 <== mds.0 v2:172.21.15.129:6838/4208792665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5616ef99e080 con 0x5616efab3990 2024-04-28T12:51:05.976 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42a1600640 10 client.25232 handle_command_reply: tid=0 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 client.25232 shutdown 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 -- 172.21.15.106:0/2126731046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 msgr2=0x5616efa42dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa42dd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42ac099690 tx=0x7f42ac00efd0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 -- 172.21.15.106:0/2126731046 shutdown_connections 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5616efab3990 0x5616efab04a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616efa37e20 0x5616efa42dd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42a1600640 0 client.25232 ms_handle_reset on v2:172.21.15.129:6838/4208792665 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5616efa32cb0 0x5616efa3fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 --2- 172.21.15.106:0/2126731046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5616efa322c0 0x5616efa3f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 -- 172.21.15.106:0/2126731046 >> 172.21.15.106:0/2126731046 conn(0x5616ef9c6d10 msgr2=0x5616efa2c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 -- 172.21.15.106:0/2126731046 shutdown_connections 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 1 -- 172.21.15.106:0/2126731046 wait complete. 2024-04-28T12:51:05.977 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.976+0000 7f42bd78e740 20 client.25232 trim_cache size 0 max 16384 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24578, 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 130.872080705, 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 137.23187964600001 2024-04-28T12:51:05.979 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:05.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 -- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3c40 msgr2=0x7f42980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.979 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3c40 0x7f42980a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5616efa33510 tx=0x7f429c005850 comp rx=0 tx=0).stop 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 -- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4104320 msgr2=0x7f42b41a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4104320 0x7f42b41a2160 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00eb10 tx=0x7f42ac00c2d0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 -- 172.21.15.106:0/2550862906 shutdown_connections 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f42980a3c40 0x7f42980a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42b4104320 0x7f42b41a2160 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42b4103960 0x7f42b419b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 --2- 172.21.15.106:0/2550862906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42b4102770 0x7f42b419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 -- 172.21.15.106:0/2550862906 >> 172.21.15.106:0/2550862906 conn(0x7f42b40fdce0 msgr2=0x7f42b4074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 -- 172.21.15.106:0/2550862906 shutdown_connections 2024-04-28T12:51:05.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:05.977+0000 7f42b9200640 1 -- 172.21.15.106:0/2550862906 wait complete. 2024-04-28T12:51:05.992 INFO:tasks.cephadm:Setup complete, yielding 2024-04-28T12:51:05.993 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-28T12:51:06.001 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi106.front.sepia.ceph.com 2024-04-28T12:51:06.002 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-28T12:51:06.430 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:51:07.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:07 smithi171 ceph-mon[63127]: pgmap v388: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 114 B/s wr, 9 op/s; 13 B/s, 0 objects/s recovering 2024-04-28T12:51:07.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:07 smithi129 ceph-mon[62418]: pgmap v388: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 114 B/s wr, 9 op/s; 13 B/s, 0 objects/s recovering 2024-04-28T12:51:07.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:07 smithi106 ceph-mon[28081]: pgmap v388: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 114 B/s wr, 9 op/s; 13 B/s, 0 objects/s recovering 2024-04-28T12:51:08.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.584+0000 7fcab8e00700 1 -- 172.21.15.106:0/4221277467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcab4105030 msgr2=0x7fcab410b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:08.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.584+0000 7fcab8e00700 1 --2- 172.21.15.106:0/4221277467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcab4105030 0x7fcab410b8e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcaac00b0d0 tx=0x7fcaac00b3e0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.584+0000 7fcab8e00700 1 -- 172.21.15.106:0/4221277467 shutdown_connections 2024-04-28T12:51:08.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.584+0000 7fcab8e00700 1 --2- 172.21.15.106:0/4221277467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcab4105030 0x7fcab410b8e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.584+0000 7fcab8e00700 1 --2- 172.21.15.106:0/4221277467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcab4103490 0x7fcab41038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.584+0000 7fcab8e00700 1 --2- 172.21.15.106:0/4221277467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcab4104690 0x7fcab4104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.584+0000 7fcab8e00700 1 -- 172.21.15.106:0/4221277467 >> 172.21.15.106:0/4221277467 conn(0x7fcab40fe9f0 msgr2=0x7fcab4100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:08.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.584+0000 7fcab8e00700 1 -- 172.21.15.106:0/4221277467 shutdown_connections 2024-04-28T12:51:08.583 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.585+0000 7fcab8e00700 1 -- 172.21.15.106:0/4221277467 wait complete. 2024-04-28T12:51:08.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.586+0000 7fcab8e00700 1 Processor -- start 2024-04-28T12:51:08.584 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.586+0000 7fcab8e00700 1 -- start start 2024-04-28T12:51:08.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.586+0000 7fcab8e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcab4103490 0x7fcab4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:08.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.586+0000 7fcab8e00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcab4104690 0x7fcab4199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:08.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.586+0000 7fcab8e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcab4105030 0x7fcab41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:08.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.586+0000 7fcab8e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcab410ef60 con 0x7fcab4105030 2024-04-28T12:51:08.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.586+0000 7fcab8e00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcab410ede0 con 0x7fcab4103490 2024-04-28T12:51:08.585 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.586+0000 7fcab8e00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcab410f0e0 con 0x7fcab4104690 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcab4103490 0x7fcab4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcab4103490 0x7fcab4199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:33260/0 (socket says 172.21.15.106:33260) 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcab4104690 0x7fcab4199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1e00700 1 -- 172.21.15.106:0/939488853 learned_addr learned my addr 172.21.15.106:0/939488853 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab2800700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcab4105030 0x7fcab41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1400700 1 -- 172.21.15.106:0/939488853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcab4103490 msgr2=0x7fcab4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1400700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcab4103490 0x7fcab4199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1400700 1 -- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcab4105030 msgr2=0x7fcab41a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1400700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcab4105030 0x7fcab41a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1400700 1 -- 172.21.15.106:0/939488853 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab41a1210 con 0x7fcab4104690 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab2800700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcab4105030 0x7fcab41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.587+0000 7fcab1400700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcab4104690 0x7fcab4199b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000ea20 tx=0x7fcaa000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:08.586 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.588+0000 7fcaaaa00700 1 -- 172.21.15.106:0/939488853 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0004220 con 0x7fcab4104690 2024-04-28T12:51:08.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.588+0000 7fcaaaa00700 1 -- 172.21.15.106:0/939488853 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa000ce20 con 0x7fcab4104690 2024-04-28T12:51:08.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.588+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab41a1500 con 0x7fcab4104690 2024-04-28T12:51:08.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.588+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab41a1a40 con 0x7fcab4104690 2024-04-28T12:51:08.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.588+0000 7fcaaaa00700 1 -- 172.21.15.106:0/939488853 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0003e00 con 0x7fcab4104690 2024-04-28T12:51:08.587 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.589+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab406a110 con 0x7fcab4104690 2024-04-28T12:51:08.590 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.594+0000 7fcaaaa00700 1 -- 172.21.15.106:0/939488853 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fcaa000c970 con 0x7fcab4104690 2024-04-28T12:51:08.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.595+0000 7fcaaaa00700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fca980a3cd0 0x7fca980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:08.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.595+0000 7fcaaaa00700 1 -- 172.21.15.106:0/939488853 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fcaa00d0c60 con 0x7fcab4104690 2024-04-28T12:51:08.592 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.596+0000 7fcab1e00700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fca980a3cd0 0x7fca980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:08.593 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.596+0000 7fcab1e00700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fca980a3cd0 0x7fca980a6190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005950 tx=0x7fcaa4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:08.596 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.600+0000 7fcaaaa00700 1 -- 172.21.15.106:0/939488853 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcaa009d0d0 con 0x7fcab4104690 2024-04-28T12:51:08.845 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.848+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fcab4063860 con 0x7fca980a3cd0 2024-04-28T12:51:08.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.850+0000 7fcaaaa00700 1 -- 172.21.15.106:0/939488853 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fcab4063860 con 0x7fca980a3cd0 2024-04-28T12:51:08.847 INFO:teuthology.orchestra.run.smithi106.stdout:Backend: cephadm 2024-04-28T12:51:08.847 INFO:teuthology.orchestra.run.smithi106.stdout:Available: Yes 2024-04-28T12:51:08.847 INFO:teuthology.orchestra.run.smithi106.stdout:Paused: No 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fca980a3cd0 msgr2=0x7fca980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fca980a3cd0 0x7fca980a6190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005950 tx=0x7fcaa4009260 comp rx=0 tx=0).stop 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcab4104690 msgr2=0x7fcab4199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcab4104690 0x7fcab4199b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000ea20 tx=0x7fcaa000ed30 comp rx=0 tx=0).stop 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 shutdown_connections 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcab4105030 0x7fcab41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcab4103490 0x7fcab4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fca980a3cd0 0x7fca980a6190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 --2- 172.21.15.106:0/939488853 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcab4104690 0x7fcab4199b40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 >> 172.21.15.106:0/939488853 conn(0x7fcab40fe9f0 msgr2=0x7fcab4107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:08.850 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 shutdown_connections 2024-04-28T12:51:08.851 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:08.853+0000 7fcab8e00700 1 -- 172.21.15.106:0/939488853 wait complete. 2024-04-28T12:51:09.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:09 smithi171 ceph-mon[63127]: pgmap v389: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-04-28T12:51:09.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:09 smithi129 ceph-mon[62418]: pgmap v389: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-04-28T12:51:09.526 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:09 smithi106 ceph-mon[28081]: pgmap v389: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 10 op/s; 11 B/s, 0 objects/s recovering 2024-04-28T12:51:09.586 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-28T12:51:10.023 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:51:10.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:10 smithi171 ceph-mon[63127]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:10.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:10 smithi129 ceph-mon[62418]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:10.514 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:10 smithi106 ceph-mon[28081]: from='client.25231 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:11.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:11 smithi106 ceph-mon[28081]: pgmap v390: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-28T12:51:11.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:11 smithi171 ceph-mon[63127]: pgmap v390: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-28T12:51:11.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:11 smithi129 ceph-mon[62418]: pgmap v390: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-28T12:51:12.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.200+0000 7f1dcb400700 1 -- 172.21.15.106:0/581407564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc410acc0 msgr2=0x7f1dc410b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:12.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.200+0000 7f1dcb400700 1 --2- 172.21.15.106:0/581407564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc410acc0 0x7f1dc410b120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1db40099c0 tx=0x7f1db4009cd0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.200+0000 7f1dcb400700 1 -- 172.21.15.106:0/581407564 shutdown_connections 2024-04-28T12:51:12.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.200+0000 7f1dcb400700 1 --2- 172.21.15.106:0/581407564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dc410b660 0x7f1dc4111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.200+0000 7f1dcb400700 1 --2- 172.21.15.106:0/581407564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc4109ac0 0x7f1dc4109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.200+0000 7f1dcb400700 1 --2- 172.21.15.106:0/581407564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc410acc0 0x7f1dc410b120 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.200+0000 7f1dcb400700 1 -- 172.21.15.106:0/581407564 >> 172.21.15.106:0/581407564 conn(0x7f1dc4078a90 msgr2=0x7f1dc407aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:12.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.200+0000 7f1dcb400700 1 -- 172.21.15.106:0/581407564 shutdown_connections 2024-04-28T12:51:12.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.200+0000 7f1dcb400700 1 -- 172.21.15.106:0/581407564 wait complete. 2024-04-28T12:51:12.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.201+0000 7f1dcb400700 1 Processor -- start 2024-04-28T12:51:12.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.201+0000 7f1dcb400700 1 -- start start 2024-04-28T12:51:12.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dcb400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc4109ac0 0x7f1dc419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:12.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dcb400700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dc410acc0 0x7f1dc41a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:12.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dcb400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc410b660 0x7f1dc41a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:12.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dcb400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41155a0 con 0x7f1dc4109ac0 2024-04-28T12:51:12.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dcb400700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4115420 con 0x7f1dc410acc0 2024-04-28T12:51:12.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dcb400700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4115720 con 0x7f1dc410b660 2024-04-28T12:51:12.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc8a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc4109ac0 0x7f1dc419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc8a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc4109ac0 0x7f1dc419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56480/0 (socket says 172.21.15.106:56480) 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc9400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc410b660 0x7f1dc41a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc8a00700 1 -- 172.21.15.106:0/4176475155 learned_addr learned my addr 172.21.15.106:0/4176475155 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc3e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dc410acc0 0x7f1dc41a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc8a00700 1 -- 172.21.15.106:0/4176475155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc410b660 msgr2=0x7f1dc41a71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc8a00700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc410b660 0x7f1dc41a71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc8a00700 1 -- 172.21.15.106:0/4176475155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dc410acc0 msgr2=0x7f1dc41a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc8a00700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dc410acc0 0x7f1dc41a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.202+0000 7f1dc8a00700 1 -- 172.21.15.106:0/4176475155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc41a7840 con 0x7f1dc4109ac0 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.203+0000 7f1dc8a00700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc4109ac0 0x7f1dc419fbe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1db800e980 tx=0x7f1db800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.203+0000 7f1dc1600700 1 -- 172.21.15.106:0/4176475155 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8004230 con 0x7f1dc4109ac0 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.203+0000 7f1dc1600700 1 -- 172.21.15.106:0/4176475155 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db80043d0 con 0x7f1dc4109ac0 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.203+0000 7f1dc1600700 1 -- 172.21.15.106:0/4176475155 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8003d40 con 0x7f1dc4109ac0 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.203+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc41a7b30 con 0x7f1dc4109ac0 2024-04-28T12:51:12.201 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.203+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a8020 con 0x7f1dc4109ac0 2024-04-28T12:51:12.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.204+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc406a110 con 0x7f1dc4109ac0 2024-04-28T12:51:12.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.206+0000 7f1dc1600700 1 -- 172.21.15.106:0/4176475155 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1db800c810 con 0x7f1dc4109ac0 2024-04-28T12:51:12.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.208+0000 7f1dc1600700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dac0a3d50 0x7f1dac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:12.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.208+0000 7f1dc1600700 1 -- 172.21.15.106:0/4176475155 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f1db80d0ae0 con 0x7f1dc4109ac0 2024-04-28T12:51:12.204 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.208+0000 7f1dc3e00700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dac0a3d50 0x7f1dac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:12.205 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.209+0000 7f1dc3e00700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dac0a3d50 0x7f1dac0a6210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:12.209 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.212+0000 7f1dc1600700 1 -- 172.21.15.106:0/4176475155 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f1db809cf50 con 0x7f1dc4109ac0 2024-04-28T12:51:12.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.455+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1dc4063860 con 0x7f1dac0a3d50 2024-04-28T12:51:12.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.476+0000 7f1dc1600700 1 -- 172.21.15.106:0/4176475155 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f1dc4063860 con 0x7f1dac0a3d50 2024-04-28T12:51:12.473 INFO:teuthology.orchestra.run.smithi106.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.a smithi106 running (2m) 26s ago 2m 16.5M - 19.0.0-3306-gbc612ffb 55dc54019ad8 03e72f101791 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.b smithi129 running (2m) 75s ago 2m 25.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 86d9c5e1c4da 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.c smithi171 running (2m) 75s ago 2m 18.0M - 19.0.0-3306-gbc612ffb 55dc54019ad8 d34769c4a5f8 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.d smithi106 running (2m) 26s ago 2m 15.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 058a2071450b 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.e smithi129 running (2m) 75s ago 2m 18.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 958082fb42cb 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.f smithi171 running (2m) 75s ago 2m 18.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 82ae24370006 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.g smithi106 running (2m) 26s ago 2m 16.2M - 19.0.0-3306-gbc612ffb 55dc54019ad8 9adac8f9ea2c 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.h smithi129 running (2m) 75s ago 2m 16.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 4b77b95139ba 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.i smithi171 running (2m) 75s ago 2m 16.9M - 19.0.0-3306-gbc612ffb 55dc54019ad8 56b0f9f78324 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.j smithi106 running (2m) 26s ago 2m 15.3M - 19.0.0-3306-gbc612ffb 55dc54019ad8 4f9290cb23b5 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.k smithi129 running (2m) 75s ago 2m 14.0M - 19.0.0-3306-gbc612ffb 55dc54019ad8 3ebade331ac7 2024-04-28T12:51:12.474 INFO:teuthology.orchestra.run.smithi106.stdout:mds.l smithi171 running (3m) 75s ago 3m 16.4M - 19.0.0-3306-gbc612ffb 55dc54019ad8 758fa61b1c1e 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:mgr.x smithi106 *:9283,8765 running (11m) 26s ago 11m 515M - 19.0.0-3306-gbc612ffb 55dc54019ad8 489a4dc1e7dc 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:mgr.y smithi129 *:8443,8765 running (8m) 75s ago 8m 454M - 19.0.0-3306-gbc612ffb 55dc54019ad8 2f3649a5e5d2 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:mgr.z smithi171 *:8443,8765 running (8m) 75s ago 8m 455M - 19.0.0-3306-gbc612ffb 55dc54019ad8 8c848c386a46 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:mon.a smithi106 running (11m) 26s ago 11m 46.7M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 d6af359f5189 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:mon.b smithi129 running (9m) 75s ago 9m 41.0M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 ff947229bf24 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:mon.c smithi171 running (9m) 75s ago 9m 41.7M 2048M 19.0.0-3306-gbc612ffb 55dc54019ad8 997c6c7c6a4e 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:osd.0 smithi106 running (8m) 26s ago 8m 53.3M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 08c2d668e8f2 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:osd.1 smithi129 running (8m) 75s ago 8m 56.5M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 e3eaf3eede8f 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:osd.2 smithi171 running (7m) 75s ago 7m 46.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 3220d67fb743 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:osd.3 smithi106 running (7m) 26s ago 7m 43.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 e5912e529bb5 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:osd.4 smithi129 running (6m) 75s ago 6m 52.1M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 5c896d29b2ef 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:osd.5 smithi171 running (6m) 75s ago 6m 51.6M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 0bc73af7e636 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:osd.6 smithi106 running (5m) 26s ago 5m 41.3M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 809fb8ebcacd 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:osd.7 smithi129 running (5m) 75s ago 5m 44.7M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 733a888307ad 2024-04-28T12:51:12.475 INFO:teuthology.orchestra.run.smithi106.stdout:osd.8 smithi171 running (4m) 75s ago 4m 44.5M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 eebdbc9b6b33 2024-04-28T12:51:12.476 INFO:teuthology.orchestra.run.smithi106.stdout:osd.9 smithi106 running (4m) 26s ago 4m 42.1M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 acb5a1a30184 2024-04-28T12:51:12.476 INFO:teuthology.orchestra.run.smithi106.stdout:osd.10 smithi129 running (3m) 75s ago 3m 44.4M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 670606a19db6 2024-04-28T12:51:12.476 INFO:teuthology.orchestra.run.smithi106.stdout:osd.11 smithi171 running (3m) 75s ago 3m 48.2M 4272M 19.0.0-3306-gbc612ffb 55dc54019ad8 f7364d4bcd90 2024-04-28T12:51:12.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dac0a3d50 msgr2=0x7f1dac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:12.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dac0a3d50 0x7f1dac0a6210 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1db4000c00 tx=0x7f1db4005fb0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc4109ac0 msgr2=0x7f1dc419fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:12.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc4109ac0 0x7f1dc419fbe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1db800e980 tx=0x7f1db800ec90 comp rx=0 tx=0).stop 2024-04-28T12:51:12.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 shutdown_connections 2024-04-28T12:51:12.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1dac0a3d50 0x7f1dac0a6210 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dc410b660 0x7f1dc41a71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc4109ac0 0x7f1dc419fbe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 --2- 172.21.15.106:0/4176475155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dc410acc0 0x7f1dc41a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:12.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.480+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 >> 172.21.15.106:0/4176475155 conn(0x7f1dc4078a90 msgr2=0x7f1dc410dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:12.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.481+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 shutdown_connections 2024-04-28T12:51:12.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:12.481+0000 7f1dcb400700 1 -- 172.21.15.106:0/4176475155 wait complete. 2024-04-28T12:51:13.254 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-28T12:51:13.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:13 smithi171 ceph-mon[63127]: pgmap v391: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T12:51:13.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:13 smithi129 ceph-mon[62418]: pgmap v391: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T12:51:13.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:13 smithi106 ceph-mon[28081]: pgmap v391: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T12:51:13.698 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:51:14.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:14 smithi171 ceph-mon[63127]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:14.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:14 smithi129 ceph-mon[62418]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:14.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:14 smithi106 ceph-mon[28081]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:15.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:15 smithi171 ceph-mon[63127]: pgmap v392: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-28T12:51:15.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:15 smithi129 ceph-mon[62418]: pgmap v392: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-28T12:51:15.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:15 smithi106 ceph-mon[28081]: pgmap v392: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-28T12:51:15.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.776+0000 7f7e84c00700 1 -- 172.21.15.106:0/2206693574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 msgr2=0x7f7e80109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:15.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.776+0000 7f7e84c00700 1 --2- 172.21.15.106:0/2206693574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 0x7f7e80109dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7e700099c0 tx=0x7f7e70009cd0 comp rx=0 tx=0).stop 2024-04-28T12:51:15.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.776+0000 7f7e84c00700 1 -- 172.21.15.106:0/2206693574 shutdown_connections 2024-04-28T12:51:15.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.777+0000 7f7e84c00700 1 --2- 172.21.15.106:0/2206693574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e8010b550 0x7f7e80111e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:15.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.777+0000 7f7e84c00700 1 --2- 172.21.15.106:0/2206693574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 0x7f7e80109dd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:15.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.777+0000 7f7e84c00700 1 --2- 172.21.15.106:0/2206693574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8010abb0 0x7f7e8010b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:15.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.777+0000 7f7e84c00700 1 -- 172.21.15.106:0/2206693574 >> 172.21.15.106:0/2206693574 conn(0x7f7e80078980 msgr2=0x7f7e8007adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:15.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.777+0000 7f7e84c00700 1 -- 172.21.15.106:0/2206693574 shutdown_connections 2024-04-28T12:51:15.774 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.777+0000 7f7e84c00700 1 -- 172.21.15.106:0/2206693574 wait complete. 2024-04-28T12:51:15.775 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.778+0000 7f7e84c00700 1 Processor -- start 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.778+0000 7f7e84c00700 1 -- start start 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.778+0000 7f7e84c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 0x7f7e8019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.778+0000 7f7e84c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e8010abb0 0x7f7e801a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.778+0000 7f7e84c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8010b550 0x7f7e801a7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.778+0000 7f7e84c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80115440 con 0x7f7e801099b0 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.778+0000 7f7e84c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801152c0 con 0x7f7e8010b550 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.778+0000 7f7e84c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801155c0 con 0x7f7e8010abb0 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7f400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 0x7f7e8019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7ea00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e8010abb0 0x7f7e801a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7f400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 0x7f7e8019fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56512/0 (socket says 172.21.15.106:56512) 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7f400700 1 -- 172.21.15.106:0/3605071309 learned_addr learned my addr 172.21.15.106:0/3605071309 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:15.776 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7fe00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8010b550 0x7f7e801a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:15.777 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7ea00700 1 -- 172.21.15.106:0/3605071309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8010b550 msgr2=0x7f7e801a7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7ea00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8010b550 0x7f7e801a7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7ea00700 1 -- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 msgr2=0x7f7e8019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7ea00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 0x7f7e8019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7ea00700 1 -- 172.21.15.106:0/3605071309 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e801a7710 con 0x7f7e8010abb0 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7f400700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 0x7f7e8019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.779+0000 7f7e7ea00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e8010abb0 0x7f7e801a0040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00ea20 tx=0x7f7e6c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.780+0000 7f7e77e00700 1 -- 172.21.15.106:0/3605071309 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c00cbd0 con 0x7f7e8010abb0 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.780+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e801a7a00 con 0x7f7e8010abb0 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.780+0000 7f7e77e00700 1 -- 172.21.15.106:0/3605071309 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e6c00ee60 con 0x7f7e8010abb0 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.780+0000 7f7e77e00700 1 -- 172.21.15.106:0/3605071309 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c018680 con 0x7f7e8010abb0 2024-04-28T12:51:15.778 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.780+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e801a7f90 con 0x7f7e8010abb0 2024-04-28T12:51:15.779 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.781+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8006a110 con 0x7f7e8010abb0 2024-04-28T12:51:15.780 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.784+0000 7f7e77e00700 1 -- 172.21.15.106:0/3605071309 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7e6c010430 con 0x7f7e8010abb0 2024-04-28T12:51:15.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.785+0000 7f7e77e00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7e640a3d50 0x7f7e640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:15.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.785+0000 7f7e77e00700 1 -- 172.21.15.106:0/3605071309 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f7e6c0d09d0 con 0x7f7e8010abb0 2024-04-28T12:51:15.782 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.786+0000 7f7e7f400700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7e640a3d50 0x7f7e640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:15.783 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.786+0000 7f7e7f400700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7e640a3d50 0x7f7e640a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7e70000c00 tx=0x7f7e70005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:15.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:15.790+0000 7f7e77e00700 1 -- 172.21.15.106:0/3605071309 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7e6c09ce40 con 0x7f7e8010abb0 2024-04-28T12:51:16.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.027+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7e80063860 con 0x7f7e640a3d50 2024-04-28T12:51:16.041 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.044+0000 7f7e77e00700 1 -- 172.21.15.106:0/3605071309 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7e80063860 con 0x7f7e640a3d50 2024-04-28T12:51:16.042 INFO:teuthology.orchestra.run.smithi106.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-28T12:51:16.042 INFO:teuthology.orchestra.run.smithi106.stdout:mds.all 12/12 79s ago 3m smithi106=a;smithi106=d;smithi106=g;smithi106=j;smithi129=b;smithi129=e;smithi129=h;smithi129=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-04-28T12:51:16.042 INFO:teuthology.orchestra.run.smithi106.stdout:mgr 3/3 79s ago 9m smithi106=x;smithi129=y;smithi171=z;count:3 2024-04-28T12:51:16.042 INFO:teuthology.orchestra.run.smithi106.stdout:mon 3/3 79s ago 9m smithi106:172.21.15.106=a;smithi129:172.21.15.129=b;smithi171:172.21.15.171=c;count:3 2024-04-28T12:51:16.042 INFO:teuthology.orchestra.run.smithi106.stdout:osd 12 79s ago - 2024-04-28T12:51:16.044 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7e640a3d50 msgr2=0x7f7e640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7e640a3d50 0x7f7e640a6210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7e70000c00 tx=0x7f7e70005fb0 comp rx=0 tx=0).stop 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e8010abb0 msgr2=0x7f7e801a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e8010abb0 0x7f7e801a0040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00ea20 tx=0x7f7e6c00ed30 comp rx=0 tx=0).stop 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 shutdown_connections 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e8010b550 0x7f7e801a7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e801099b0 0x7f7e8019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f7e640a3d50 0x7f7e640a6210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 --2- 172.21.15.106:0/3605071309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e8010abb0 0x7f7e801a0040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.047+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 >> 172.21.15.106:0/3605071309 conn(0x7f7e80078980 msgr2=0x7f7e8010deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.048+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 shutdown_connections 2024-04-28T12:51:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:16.048+0000 7f7e84c00700 1 -- 172.21.15.106:0/3605071309 wait complete. 2024-04-28T12:51:16.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:16 smithi106 ceph-mon[28081]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:16.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:16 smithi106 ceph-mon[28081]: pgmap v393: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-28T12:51:16.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:16 smithi129 ceph-mon[62418]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:16.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:16 smithi129 ceph-mon[62418]: pgmap v393: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-28T12:51:16.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:16 smithi171 ceph-mon[63127]: from='client.25243 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:16.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:16 smithi171 ceph-mon[63127]: pgmap v393: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-04-28T12:51:16.850 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-28T12:51:17.297 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:51:19.262 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:19 smithi106 ceph-mon[28081]: pgmap v394: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:51:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.433+0000 7f9927000700 1 -- 172.21.15.106:0/2836131697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99201056c0 msgr2=0x7f9920105ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.433+0000 7f9927000700 1 --2- 172.21.15.106:0/2836131697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99201056c0 0x7f9920105ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f99100099c0 tx=0x7f9910009cd0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.433+0000 7f9927000700 1 -- 172.21.15.106:0/2836131697 shutdown_connections 2024-04-28T12:51:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.433+0000 7f9927000700 1 --2- 172.21.15.106:0/2836131697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9920107260 0x7f992010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.433+0000 7f9927000700 1 --2- 172.21.15.106:0/2836131697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99201056c0 0x7f9920105ae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.433+0000 7f9927000700 1 --2- 172.21.15.106:0/2836131697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99201068c0 0x7f9920106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.433+0000 7f9927000700 1 -- 172.21.15.106:0/2836131697 >> 172.21.15.106:0/2836131697 conn(0x7f9920100c80 msgr2=0x7f99201030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.433+0000 7f9927000700 1 -- 172.21.15.106:0/2836131697 shutdown_connections 2024-04-28T12:51:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.433+0000 7f9927000700 1 -- 172.21.15.106:0/2836131697 wait complete. 2024-04-28T12:51:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.434+0000 7f9927000700 1 Processor -- start 2024-04-28T12:51:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.434+0000 7f9927000700 1 -- start start 2024-04-28T12:51:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.434+0000 7f9927000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99201056c0 0x7f992019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.434+0000 7f9927000700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99201068c0 0x7f992019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9927000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9920107260 0x7f99201a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9927000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9920111160 con 0x7f99201056c0 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9927000700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9920110fe0 con 0x7f99201068c0 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9927000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99201112e0 con 0x7f9920107260 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f991fe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99201056c0 0x7f992019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9917e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99201068c0 0x7f992019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f991fe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99201056c0 0x7f992019b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38220/0 (socket says 172.21.15.106:38220) 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f991fe00700 1 -- 172.21.15.106:0/784156917 learned_addr learned my addr 172.21.15.106:0/784156917 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9925000700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9920107260 0x7f99201a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9917e00700 1 -- 172.21.15.106:0/784156917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9920107260 msgr2=0x7f99201a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9917e00700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9920107260 0x7f99201a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9917e00700 1 -- 172.21.15.106:0/784156917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99201056c0 msgr2=0x7f992019b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9917e00700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99201056c0 0x7f992019b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.435+0000 7f9917e00700 1 -- 172.21.15.106:0/784156917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99201a3430 con 0x7f99201068c0 2024-04-28T12:51:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.436+0000 7f9917e00700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99201068c0 0x7f992019bd10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f990800e9f0 tx=0x7f990800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.436+0000 7f991d600700 1 -- 172.21.15.106:0/784156917 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908004220 con 0x7f99201068c0 2024-04-28T12:51:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.436+0000 7f991d600700 1 -- 172.21.15.106:0/784156917 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99080043c0 con 0x7f99201068c0 2024-04-28T12:51:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.436+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99201a3720 con 0x7f99201068c0 2024-04-28T12:51:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.436+0000 7f991d600700 1 -- 172.21.15.106:0/784156917 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908005020 con 0x7f99201068c0 2024-04-28T12:51:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.436+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201a3c10 con 0x7f99201068c0 2024-04-28T12:51:19.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.437+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f992006a110 con 0x7f99201068c0 2024-04-28T12:51:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.439+0000 7f991d600700 1 -- 172.21.15.106:0/784156917 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9908004750 con 0x7f99201068c0 2024-04-28T12:51:19.437 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.440+0000 7f991d600700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99040a3cd0 0x7f99040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:19.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.441+0000 7f991d600700 1 -- 172.21.15.106:0/784156917 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f99080d0a60 con 0x7f99201068c0 2024-04-28T12:51:19.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.441+0000 7f991fe00700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99040a3cd0 0x7f99040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:19.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.442+0000 7f991fe00700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99040a3cd0 0x7f99040a6190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9910000c00 tx=0x7f9910005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:19.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.443+0000 7f991d600700 1 -- 172.21.15.106:0/784156917 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f990809ced0 con 0x7f99201068c0 2024-04-28T12:51:19.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:19 smithi171 ceph-mon[63127]: pgmap v394: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:51:19.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:19 smithi129 ceph-mon[62418]: pgmap v394: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:51:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.677+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f9920063860 con 0x7f99040a3cd0 2024-04-28T12:51:19.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.681+0000 7f991d600700 1 -- 172.21.15.106:0/784156917 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f9920063860 con 0x7f99040a3cd0 2024-04-28T12:51:19.679 INFO:teuthology.orchestra.run.smithi106.stdout:HOST ADDR LABELS STATUS 2024-04-28T12:51:19.679 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106 172.21.15.106 2024-04-28T12:51:19.679 INFO:teuthology.orchestra.run.smithi106.stdout:smithi129 172.21.15.129 2024-04-28T12:51:19.679 INFO:teuthology.orchestra.run.smithi106.stdout:smithi171 172.21.15.171 2024-04-28T12:51:19.679 INFO:teuthology.orchestra.run.smithi106.stdout:3 hosts in cluster 2024-04-28T12:51:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.684+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99040a3cd0 msgr2=0x7f99040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.684+0000 7f9927000700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99040a3cd0 0x7f99040a6190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9910000c00 tx=0x7f9910005fb0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99201068c0 msgr2=0x7f992019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99201068c0 0x7f992019bd10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f990800e9f0 tx=0x7f990800ed00 comp rx=0 tx=0).stop 2024-04-28T12:51:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 shutdown_connections 2024-04-28T12:51:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f99040a3cd0 0x7f99040a6190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9920107260 0x7f99201a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99201056c0 0x7f992019b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 --2- 172.21.15.106:0/784156917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99201068c0 0x7f992019bd10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:19.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 >> 172.21.15.106:0/784156917 conn(0x7f9920100c80 msgr2=0x7f9920109be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:19.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 shutdown_connections 2024-04-28T12:51:19.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:19.685+0000 7f9927000700 1 -- 172.21.15.106:0/784156917 wait complete. 2024-04-28T12:51:20.471 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-28T12:51:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:51:21.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:21 smithi106 ceph-mon[28081]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:21.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:21 smithi106 ceph-mon[28081]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-28T12:51:21.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:21 smithi171 ceph-mon[63127]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:21.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:21 smithi171 ceph-mon[63127]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-28T12:51:21.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:21 smithi129 ceph-mon[62418]: from='client.25256 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:21.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:21 smithi129 ceph-mon[62418]: pgmap v395: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-28T12:51:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.094+0000 7f2572200700 1 -- 172.21.15.106:0/2777309558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f256c1045b0 msgr2=0x7f256c104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.094+0000 7f2572200700 1 --2- 172.21.15.106:0/2777309558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f256c1045b0 0x7f256c104a10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2558009920 tx=0x7f2558009c30 comp rx=0 tx=0).stop 2024-04-28T12:51:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.094+0000 7f2572200700 1 -- 172.21.15.106:0/2777309558 shutdown_connections 2024-04-28T12:51:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.094+0000 7f2572200700 1 --2- 172.21.15.106:0/2777309558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f256c104f50 0x7f256c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.094+0000 7f2572200700 1 --2- 172.21.15.106:0/2777309558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f256c1033b0 0x7f256c1037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.094+0000 7f2572200700 1 --2- 172.21.15.106:0/2777309558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f256c1045b0 0x7f256c104a10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.094+0000 7f2572200700 1 -- 172.21.15.106:0/2777309558 >> 172.21.15.106:0/2777309558 conn(0x7f256c0fe950 msgr2=0x7f256c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:23.091 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.094+0000 7f2572200700 1 -- 172.21.15.106:0/2777309558 shutdown_connections 2024-04-28T12:51:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.094+0000 7f2572200700 1 -- 172.21.15.106:0/2777309558 wait complete. 2024-04-28T12:51:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.095+0000 7f2572200700 1 Processor -- start 2024-04-28T12:51:23.092 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.095+0000 7f2572200700 1 -- start start 2024-04-28T12:51:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f2572200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f256c1033b0 0x7f256c19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f2572200700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f256c1045b0 0x7f256c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f2572200700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f256c104f50 0x7f256c1a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f2572200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f256c10eea0 con 0x7f256c1033b0 2024-04-28T12:51:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f2572200700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f256c10ed20 con 0x7f256c104f50 2024-04-28T12:51:23.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f2572200700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f256c10f020 con 0x7f256c1045b0 2024-04-28T12:51:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f256be00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f256c1045b0 0x7f256c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f2570e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f256c1033b0 0x7f256c19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f256be00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f256c1045b0 0x7f256c19bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:47880/0 (socket says 172.21.15.106:47880) 2024-04-28T12:51:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f256be00700 1 -- 172.21.15.106:0/4025035893 learned_addr learned my addr 172.21.15.106:0/4025035893 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:23.094 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.096+0000 7f2571800700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f256c104f50 0x7f256c1a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2570e00700 1 -- 172.21.15.106:0/4025035893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f256c1045b0 msgr2=0x7f256c19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2570e00700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f256c1045b0 0x7f256c19bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2570e00700 1 -- 172.21.15.106:0/4025035893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f256c104f50 msgr2=0x7f256c1a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2570e00700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f256c104f50 0x7f256c1a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2570e00700 1 -- 172.21.15.106:0/4025035893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c1a33c0 con 0x7f256c1033b0 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2571800700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f256c104f50 0x7f256c1a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2570e00700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f256c1033b0 0x7f256c19b760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f256000e9c0 tx=0x7f256000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2569600700 1 -- 172.21.15.106:0/4025035893 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560004230 con 0x7f256c1033b0 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2569600700 1 -- 172.21.15.106:0/4025035893 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25600043d0 con 0x7f256c1033b0 2024-04-28T12:51:23.095 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c1a36b0 con 0x7f256c1033b0 2024-04-28T12:51:23.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.097+0000 7f2569600700 1 -- 172.21.15.106:0/4025035893 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560003bc0 con 0x7f256c1033b0 2024-04-28T12:51:23.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.098+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c1a3ba0 con 0x7f256c1033b0 2024-04-28T12:51:23.096 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.099+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f256c06a110 con 0x7f256c1033b0 2024-04-28T12:51:23.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.101+0000 7f2569600700 1 -- 172.21.15.106:0/4025035893 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f256000c7c0 con 0x7f256c1033b0 2024-04-28T12:51:23.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.102+0000 7f2569600700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f25540a3cd0 0x7f25540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:23.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.102+0000 7f2569600700 1 -- 172.21.15.106:0/4025035893 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f25600d09a0 con 0x7f256c1033b0 2024-04-28T12:51:23.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.102+0000 7f256be00700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f25540a3cd0 0x7f25540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:23.099 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.103+0000 7f256be00700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f25540a3cd0 0x7f25540a6190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f255800faf0 tx=0x7f2558019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:23.103 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.107+0000 7f2569600700 1 -- 172.21.15.106:0/4025035893 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f256009ce50 con 0x7f256c1033b0 2024-04-28T12:51:23.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.352+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f256c1a3e60 con 0x7f25540a3cd0 2024-04-28T12:51:23.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.355+0000 7f2569600700 1 -- 172.21.15.106:0/4025035893 <== mgr.14152 v2:172.21.15.106:6800/1556498064 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f256c1a3e60 con 0x7f25540a3cd0 2024-04-28T12:51:23.352 INFO:teuthology.orchestra.run.smithi106.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-28T12:51:23.352 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013R400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-28T12:51:23.352 INFO:teuthology.orchestra.run.smithi106.stdout:smithi129 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300HY400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-28T12:51:23.352 INFO:teuthology.orchestra.run.smithi106.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-28T12:51:23.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f25540a3cd0 msgr2=0x7f25540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:23.355 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f25540a3cd0 0x7f25540a6190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f255800faf0 tx=0x7f2558019040 comp rx=0 tx=0).stop 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f256c1033b0 msgr2=0x7f256c19b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f256c1033b0 0x7f256c19b760 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f256000e9c0 tx=0x7f256000ecd0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 shutdown_connections 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f256c104f50 0x7f256c1a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f256c1033b0 0x7f256c19b760 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f25540a3cd0 0x7f25540a6190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 --2- 172.21.15.106:0/4025035893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f256c1045b0 0x7f256c19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.358+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 >> 172.21.15.106:0/4025035893 conn(0x7f256c0fe950 msgr2=0x7f256c075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.359+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 shutdown_connections 2024-04-28T12:51:23.356 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:23.359+0000 7f2572200700 1 -- 172.21.15.106:0/4025035893 wait complete. 2024-04-28T12:51:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:23 smithi171 ceph-mon[63127]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T12:51:23.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:23 smithi129 ceph-mon[62418]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T12:51:23.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:23 smithi106 ceph-mon[28081]: pgmap v396: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T12:51:24.123 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-28T12:51:24.132 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi106.front.sepia.ceph.com 2024-04-28T12:51:24.132 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-28T12:51:24.596 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:51:25.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:25 smithi171 ceph-mon[63127]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:25.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:25 smithi171 ceph-mon[63127]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:51:25.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:25 smithi129 ceph-mon[62418]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:25.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:25 smithi129 ceph-mon[62418]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:51:25.564 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:25 smithi106 ceph-mon[28081]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:25.564 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:25 smithi106 ceph-mon[28081]: pgmap v397: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:51:26.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.690+0000 7f7569800700 1 -- 172.21.15.106:0/1865871764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 msgr2=0x7f75641038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:26.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.690+0000 7f7569800700 1 --2- 172.21.15.106:0/1865871764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 0x7f75641038b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f75540099c0 tx=0x7f7554009cd0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.690+0000 7f7569800700 1 -- 172.21.15.106:0/1865871764 shutdown_connections 2024-04-28T12:51:26.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.690+0000 7f7569800700 1 --2- 172.21.15.106:0/1865871764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564105030 0x7f756410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.690+0000 7f7569800700 1 --2- 172.21.15.106:0/1865871764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 0x7f75641038b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.690+0000 7f7569800700 1 --2- 172.21.15.106:0/1865871764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564104690 0x7f7564104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.690+0000 7f7569800700 1 -- 172.21.15.106:0/1865871764 >> 172.21.15.106:0/1865871764 conn(0x7f75640fe9f0 msgr2=0x7f7564100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:26.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.690+0000 7f7569800700 1 -- 172.21.15.106:0/1865871764 shutdown_connections 2024-04-28T12:51:26.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.690+0000 7f7569800700 1 -- 172.21.15.106:0/1865871764 wait complete. 2024-04-28T12:51:26.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.691+0000 7f7569800700 1 Processor -- start 2024-04-28T12:51:26.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.691+0000 7f7569800700 1 -- start start 2024-04-28T12:51:26.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7569800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 0x7f756419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:26.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7569800700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564104690 0x7f756419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:26.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7569800700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564105030 0x7f75641a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:26.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7569800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f756410ef90 con 0x7f7564103490 2024-04-28T12:51:26.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7569800700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f756410ee10 con 0x7f7564105030 2024-04-28T12:51:26.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7569800700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f756410f110 con 0x7f7564104690 2024-04-28T12:51:26.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7562a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 0x7f756419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:26.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7562a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 0x7f756419b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38268/0 (socket says 172.21.15.106:38268) 2024-04-28T12:51:26.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7562a00700 1 -- 172.21.15.106:0/1837777379 learned_addr learned my addr 172.21.15.106:0/1837777379 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:26.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7562000700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564104690 0x7f756419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:26.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.692+0000 7f7563400700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564105030 0x7f75641a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:26.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f7562a00700 1 -- 172.21.15.106:0/1837777379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564104690 msgr2=0x7f756419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:26.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f7562a00700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564104690 0x7f756419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f7562a00700 1 -- 172.21.15.106:0/1837777379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564105030 msgr2=0x7f75641a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:26.691 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f7562a00700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564105030 0x7f75641a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f7562a00700 1 -- 172.21.15.106:0/1837777379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75641a34b0 con 0x7f7564103490 2024-04-28T12:51:26.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f7562a00700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 0x7f756419b850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7554003e60 tx=0x7f7554003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:26.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f755b400700 1 -- 172.21.15.106:0/1837777379 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554009e60 con 0x7f7564103490 2024-04-28T12:51:26.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f755b400700 1 -- 172.21.15.106:0/1837777379 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755400f460 con 0x7f7564103490 2024-04-28T12:51:26.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f755b400700 1 -- 172.21.15.106:0/1837777379 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7554017650 con 0x7f7564103490 2024-04-28T12:51:26.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.693+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75641a3740 con 0x7f7564103490 2024-04-28T12:51:26.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.694+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75641a3b40 con 0x7f7564103490 2024-04-28T12:51:26.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.695+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f756406a150 con 0x7f7564103490 2024-04-28T12:51:26.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.697+0000 7f755b400700 1 -- 172.21.15.106:0/1837777379 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f755400f910 con 0x7f7564103490 2024-04-28T12:51:26.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.698+0000 7f755b400700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75480a3d50 0x7f75480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:26.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.698+0000 7f755b400700 1 -- 172.21.15.106:0/1837777379 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f75540d1740 con 0x7f7564103490 2024-04-28T12:51:26.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.698+0000 7f7562000700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75480a3d50 0x7f75480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:26.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.699+0000 7f7562000700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75480a3d50 0x7f75480a6210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7550004630 tx=0x7f755000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:26.699 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.703+0000 7f755b400700 1 -- 172.21.15.106:0/1837777379 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f7554013070 con 0x7f7564103490 2024-04-28T12:51:26.974 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.977+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f75641a4060 con 0x7f7564103490 2024-04-28T12:51:26.975 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.978+0000 7f755b400700 1 -- 172.21.15.106:0/1837777379 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3106 (secure 0 0 0) 0x7f755409dc40 con 0x7f7564103490 2024-04-28T12:51:26.975 INFO:teuthology.orchestra.run.smithi106.stdout:e26 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:legacy client fscid: 1 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:Filesystem 'cephfs' (1) 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:fs_name cephfs 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:epoch 26 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:created 2024-04-28T12:48:54.598061+0000 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:modified 2024-04-28T12:49:30.023676+0000 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:tableserver 0 2024-04-28T12:51:26.976 INFO:teuthology.orchestra.run.smithi106.stdout:root 0 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:session_timeout 300 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:session_autoclose 300 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:max_file_size 1099511627776 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:max_xattr_size 65536 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:required_client_features {} 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:last_failure 0 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:last_failure_osd_epoch 0 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:max_mds 5 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:in 0,1,2,3,4 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:up {0=24578,1=24566,2=24548,3=24538,4=24520} 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:failed 2024-04-28T12:51:26.977 INFO:teuthology.orchestra.run.smithi106.stdout:damaged 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:stopped 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:data_pools [3,4] 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:metadata_pool 2 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:inline_data disabled 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:balancer 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:bal_rank_mask -1 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:standby_count_wanted 0 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:qdb_cluster leader: 24578 members: 24548,24520,24578,24538,24566 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b{0:24578} state up:active seq 3 addr [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.h{0:24518} state up:standby-replay seq 1 addr [v2:172.21.15.129:6836/1065884016,v1:172.21.15.129:6837/1065884016] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.978 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a{1:24566} state up:active seq 12 addr [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.f{1:24508} state up:standby-replay seq 1 addr [v2:172.21.15.171:6834/4221893526,v1:172.21.15.171:6835/4221893526] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.g{2:24548} state up:active seq 13 addr [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.e{2:24491} state up:standby-replay seq 1 addr [v2:172.21.15.129:6834/1242652051,v1:172.21.15.129:6835/1242652051] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.i{3:24538} state up:active seq 14 addr [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d{3:24487} state up:standby-replay seq 1 addr [v2:172.21.15.106:6836/4131098763,v1:172.21.15.106:6837/4131098763] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c{4:24520} state up:active seq 16 addr [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.j{4:24469} state up:standby-replay seq 1 addr [v2:172.21.15.106:6834/3035871722,v1:172.21.15.106:6835/3035871722] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:Standby daemons: 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.k{-1:24443} state up:standby seq 1 addr [v2:172.21.15.129:6832/2537709936,v1:172.21.15.129:6833/2537709936] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.979 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.l{-1:24445} state up:standby seq 1 addr [v2:172.21.15.171:6832/1724808557,v1:172.21.15.171:6833/1724808557] compat {c=[1],r=[1],i=[1fff]}] 2024-04-28T12:51:26.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.981+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75480a3d50 msgr2=0x7f75480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:26.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.981+0000 7f7569800700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75480a3d50 0x7f75480a6210 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7550004630 tx=0x7f755000b3c0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.981+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 msgr2=0x7f756419b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:26.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.981+0000 7f7569800700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 0x7f756419b850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7554003e60 tx=0x7f7554003e90 comp rx=0 tx=0).stop 2024-04-28T12:51:26.980 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.982+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 shutdown_connections 2024-04-28T12:51:26.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.982+0000 7f7569800700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7564105030 0x7f75641a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.982+0000 7f7569800700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7564103490 0x7f756419b850 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.982+0000 7f7569800700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f75480a3d50 0x7f75480a6210 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.982+0000 7f7569800700 1 --2- 172.21.15.106:0/1837777379 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7564104690 0x7f756419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:26.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.982+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 >> 172.21.15.106:0/1837777379 conn(0x7f75640fe9f0 msgr2=0x7f7564109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:26.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.982+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 shutdown_connections 2024-04-28T12:51:26.981 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:26.982+0000 7f7569800700 1 -- 172.21.15.106:0/1837777379 wait complete. 2024-04-28T12:51:26.982 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-28T12:51:27.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:27 smithi106 ceph-mon[28081]: pgmap v398: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:51:27.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:27 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1837777379' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-28T12:51:27.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:27 smithi171 ceph-mon[63127]: pgmap v398: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:51:27.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:27 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1837777379' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-28T12:51:27.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:27 smithi129 ceph-mon[62418]: pgmap v398: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:51:27.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:27 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1837777379' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-28T12:51:27.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-28T12:51:28.296 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:51:29.460 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:29 smithi106 ceph-mon[28081]: pgmap v399: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:51:29.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:29 smithi171 ceph-mon[63127]: pgmap v399: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:51:29.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:29 smithi129 ceph-mon[62418]: pgmap v399: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:51:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.495+0000 7faac0c00700 1 -- 172.21.15.106:0/4114878584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc10acc0 msgr2=0x7faabc10b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.495+0000 7faac0c00700 1 --2- 172.21.15.106:0/4114878584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc10acc0 0x7faabc10b120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faaa80099c0 tx=0x7faaa8009cd0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.495+0000 7faac0c00700 1 -- 172.21.15.106:0/4114878584 shutdown_connections 2024-04-28T12:51:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.495+0000 7faac0c00700 1 --2- 172.21.15.106:0/4114878584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faabc10b660 0x7faabc111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.495+0000 7faac0c00700 1 --2- 172.21.15.106:0/4114878584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faabc109ac0 0x7faabc109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.495+0000 7faac0c00700 1 --2- 172.21.15.106:0/4114878584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc10acc0 0x7faabc10b120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.495+0000 7faac0c00700 1 -- 172.21.15.106:0/4114878584 >> 172.21.15.106:0/4114878584 conn(0x7faabc078a90 msgr2=0x7faabc07aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.495+0000 7faac0c00700 1 -- 172.21.15.106:0/4114878584 shutdown_connections 2024-04-28T12:51:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.495+0000 7faac0c00700 1 -- 172.21.15.106:0/4114878584 wait complete. 2024-04-28T12:51:30.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.496+0000 7faac0c00700 1 Processor -- start 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.496+0000 7faac0c00700 1 -- start start 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faac0c00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc109ac0 0x7faabc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faac0c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faabc10acc0 0x7faabc1a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faac0c00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faabc10b660 0x7faabc1a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faac0c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faabc115530 con 0x7faabc10acc0 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faac0c00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faabc1153b0 con 0x7faabc109ac0 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faac0c00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7faabc1156b0 con 0x7faabc10b660 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faab9e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc109ac0 0x7faabc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faab9400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faabc10acc0 0x7faabc1a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faab9e00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc109ac0 0x7faabc19fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44600/0 (socket says 172.21.15.106:44600) 2024-04-28T12:51:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.497+0000 7faab9e00700 1 -- 172.21.15.106:0/2125893130 learned_addr learned my addr 172.21.15.106:0/2125893130 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faaba800700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faabc10b660 0x7faabc1a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faab9e00700 1 -- 172.21.15.106:0/2125893130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faabc10b660 msgr2=0x7faabc1a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faab9e00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faabc10b660 0x7faabc1a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faab9e00700 1 -- 172.21.15.106:0/2125893130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faabc10acc0 msgr2=0x7faabc1a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faab9e00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faabc10acc0 0x7faabc1a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faab9e00700 1 -- 172.21.15.106:0/2125893130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc1a7800 con 0x7faabc109ac0 2024-04-28T12:51:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faaba800700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faabc10b660 0x7faabc1a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faab9e00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc109ac0 0x7faabc19fba0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faaac00e970 tx=0x7faaac00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faab2a00700 1 -- 172.21.15.106:0/2125893130 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac00cbd0 con 0x7faabc109ac0 2024-04-28T12:51:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc1a7af0 con 0x7faabc109ac0 2024-04-28T12:51:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faab2a00700 1 -- 172.21.15.106:0/2125893130 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaac0044d0 con 0x7faabc109ac0 2024-04-28T12:51:30.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.498+0000 7faab2a00700 1 -- 172.21.15.106:0/2125893130 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac003df0 con 0x7faabc109ac0 2024-04-28T12:51:30.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.499+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc1a8030 con 0x7faabc109ac0 2024-04-28T12:51:30.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.500+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc06a110 con 0x7faabc109ac0 2024-04-28T12:51:30.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.502+0000 7faab2a00700 1 -- 172.21.15.106:0/2125893130 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7faaac003f90 con 0x7faabc109ac0 2024-04-28T12:51:30.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.503+0000 7faab2a00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7faaa00a3cd0 0x7faaa00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:30.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.503+0000 7faab2a00700 1 -- 172.21.15.106:0/2125893130 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7faaac0d0d90 con 0x7faabc109ac0 2024-04-28T12:51:30.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.503+0000 7faab9400700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7faaa00a3cd0 0x7faaa00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:30.500 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.504+0000 7faab9400700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7faaa00a3cd0 0x7faaa00a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faaa8000c00 tx=0x7faaa8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:30.502 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.505+0000 7faab2a00700 1 -- 172.21.15.106:0/2125893130 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7faaac09d200 con 0x7faabc109ac0 2024-04-28T12:51:30.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.733+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7faabc02e5a0 con 0x7faabc109ac0 2024-04-28T12:51:30.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.735+0000 7faab2a00700 1 -- 172.21.15.106:0/2125893130 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5737 (secure 0 0 0) 0x7faaac09cb00 con 0x7faabc109ac0 2024-04-28T12:51:30.733 INFO:teuthology.orchestra.run.smithi106.stdout:epoch 93 2024-04-28T12:51:30.733 INFO:teuthology.orchestra.run.smithi106.stdout:fsid e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T12:51:30.733 INFO:teuthology.orchestra.run.smithi106.stdout:created 2024-04-28T12:39:49.393843+0000 2024-04-28T12:51:30.733 INFO:teuthology.orchestra.run.smithi106.stdout:modified 2024-04-28T12:50:58.139109+0000 2024-04-28T12:51:30.733 INFO:teuthology.orchestra.run.smithi106.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-28T12:51:30.733 INFO:teuthology.orchestra.run.smithi106.stdout:crush_version 27 2024-04-28T12:51:30.733 INFO:teuthology.orchestra.run.smithi106.stdout:full_ratio 0.95 2024-04-28T12:51:30.733 INFO:teuthology.orchestra.run.smithi106.stdout:backfillfull_ratio 0.9 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:nearfull_ratio 0.85 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:require_min_compat_client luminous 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:min_compat_client luminous 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:require_osd_release squid 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:stretch_mode_enabled false 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:max_osd 12 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6802/3844735961,v1:172.21.15.106:6803/3844735961] [v2:172.21.15.106:6804/3844735961,v1:172.21.15.106:6805/3844735961] exists,up 0afd8792-8351-40f5-ac93-536a58b5182a 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/3542000402,v1:172.21.15.129:6801/3542000402] [v2:172.21.15.129:6802/3542000402,v1:172.21.15.129:6803/3542000402] exists,up d86a904e-10f6-402f-8107-f2fac6eead2d 2024-04-28T12:51:30.734 INFO:teuthology.orchestra.run.smithi106.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/1763026525,v1:172.21.15.171:6801/1763026525] [v2:172.21.15.171:6802/1763026525,v1:172.21.15.171:6803/1763026525] exists,up e9518ad8-2471-45eb-bf21-ab04fcd7274b 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6810/3527734792,v1:172.21.15.106:6811/3527734792] [v2:172.21.15.106:6812/3527734792,v1:172.21.15.106:6813/3527734792] exists,up 4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:osd.4 up in weight 1 up_from 33 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/2253162285,v1:172.21.15.129:6809/2253162285] [v2:172.21.15.129:6810/2253162285,v1:172.21.15.129:6811/2253162285] exists,up 505898bb-0925-4988-a527-d39888287e0e 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:osd.5 up in weight 1 up_from 38 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/1286181714,v1:172.21.15.171:6809/1286181714] [v2:172.21.15.171:6810/1286181714,v1:172.21.15.171:6811/1286181714] exists,up b4836336-5952-4041-abaf-800c8058d713 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6818/1789189109,v1:172.21.15.106:6819/1789189109] [v2:172.21.15.106:6820/1789189109,v1:172.21.15.106:6821/1789189109] exists,up 17da728b-3acb-48d3-9fda-4485aa4b3757 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6816/3741780752,v1:172.21.15.129:6817/3741780752] [v2:172.21.15.129:6818/3741780752,v1:172.21.15.129:6819/3741780752] exists,up 9d679bae-c3a9-4c6b-a604-934ae327ca2f 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:osd.8 up in weight 1 up_from 59 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/1721308305,v1:172.21.15.171:6817/1721308305] [v2:172.21.15.171:6818/1721308305,v1:172.21.15.171:6819/1721308305] exists,up 9cf9f2b4-6bf1-4740-8170-456e3b3f858a 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:osd.9 up in weight 1 up_from 67 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6826/95511724,v1:172.21.15.106:6827/95511724] [v2:172.21.15.106:6828/95511724,v1:172.21.15.106:6829/95511724] exists,up cd4886f6-691b-4475-9180-11d5262b59a5 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:osd.10 up in weight 1 up_from 72 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6824/3282354257,v1:172.21.15.129:6825/3282354257] [v2:172.21.15.129:6826/3282354257,v1:172.21.15.129:6827/3282354257] exists,up d2d8f70b-cc2e-43d1-b34f-d487b001e922 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/2516393366,v1:172.21.15.171:6825/2516393366] [v2:172.21.15.171:6826/2516393366,v1:172.21.15.171:6827/2516393366] exists,up dc45af10-78d3-4da4-b980-e7fde882529a 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 2.a [2,11] 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 2.b [2,4,8,11] 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 3.1 [4,9] 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 3.5 [3,9] 2024-04-28T12:51:30.735 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 3.6 [3,8] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 3.11 [4,7] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 3.17 [4,8] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 3.24 [4,9] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 3.37 [5,6] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 3.39 [5,6] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 4.9 [5,8] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 4.c [9,10] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 4.12 [9,10] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:pg_upmap_items 4.1f [9,8] 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:6801/2253669851 expires 2024-04-29T12:40:55.904029+0000 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:6800/2253669851 expires 2024-04-29T12:40:55.904029+0000 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:0/3545890663 expires 2024-04-29T12:40:55.904029+0000 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:0/1369056801 expires 2024-04-29T12:40:55.904029+0000 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:0/938762297 expires 2024-04-29T12:40:21.812705+0000 2024-04-28T12:51:30.736 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:0/1467729753 expires 2024-04-29T12:40:21.812705+0000 2024-04-28T12:51:30.737 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:6800/102356115 expires 2024-04-29T12:40:21.812705+0000 2024-04-28T12:51:30.737 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:0/2991131145 expires 2024-04-29T12:40:21.812705+0000 2024-04-28T12:51:30.737 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:6801/102356115 expires 2024-04-29T12:40:21.812705+0000 2024-04-28T12:51:30.737 INFO:teuthology.orchestra.run.smithi106.stdout:blocklist 172.21.15.106:0/2350345760 expires 2024-04-29T12:40:55.904029+0000 2024-04-28T12:51:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7faaa00a3cd0 msgr2=0x7faaa00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7faaa00a3cd0 0x7faaa00a6190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faaa8000c00 tx=0x7faaa8005fb0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc109ac0 msgr2=0x7faabc19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc109ac0 0x7faabc19fba0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7faaac00e970 tx=0x7faaac00ec80 comp rx=0 tx=0).stop 2024-04-28T12:51:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 shutdown_connections 2024-04-28T12:51:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7faaa00a3cd0 0x7faaa00a6190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faabc10b660 0x7faabc1a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faabc109ac0 0x7faabc19fba0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 --2- 172.21.15.106:0/2125893130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faabc10acc0 0x7faabc1a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.738+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 >> 172.21.15.106:0/2125893130 conn(0x7faabc078a90 msgr2=0x7faabc10dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.739+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 shutdown_connections 2024-04-28T12:51:30.738 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:30.739+0000 7faac0c00700 1 -- 172.21.15.106:0/2125893130 wait complete. 2024-04-28T12:51:31.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:31 smithi171 ceph-mon[63127]: pgmap v400: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:51:31.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:31 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2125893130' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-28T12:51:31.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:31 smithi129 ceph-mon[62418]: pgmap v400: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:51:31.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:31 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2125893130' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-28T12:51:31.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:31 smithi106 ceph-mon[28081]: pgmap v400: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:51:31.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:31 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2125893130' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-28T12:51:31.676 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-28T12:51:31.687 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:51:31.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.901+0000 7f736ee00640 1 -- 172.21.15.106:0/844763741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736810a180 msgr2=0x7f736810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:31.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.901+0000 7f736ee00640 1 --2- 172.21.15.106:0/844763741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736810a180 0x7f736810a600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7350009900 tx=0x7f735002a410 comp rx=0 tx=0).stop 2024-04-28T12:51:31.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.901+0000 7f736ee00640 1 -- 172.21.15.106:0/844763741 shutdown_connections 2024-04-28T12:51:31.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.901+0000 7f736ee00640 1 --2- 172.21.15.106:0/844763741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f736810ab40 0x7f73681113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:31.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.901+0000 7f736ee00640 1 --2- 172.21.15.106:0/844763741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736810a180 0x7f736810a600 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:31.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.901+0000 7f736ee00640 1 --2- 172.21.15.106:0/844763741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108f90 0x7f73681093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:31.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.901+0000 7f736ee00640 1 -- 172.21.15.106:0/844763741 >> 172.21.15.106:0/844763741 conn(0x7f7368077f40 msgr2=0x7f736807a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:31.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.901+0000 7f736ee00640 1 -- 172.21.15.106:0/844763741 shutdown_connections 2024-04-28T12:51:31.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.901+0000 7f736ee00640 1 -- 172.21.15.106:0/844763741 wait complete. 2024-04-28T12:51:31.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.902+0000 7f736ee00640 1 Processor -- start 2024-04-28T12:51:31.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.902+0000 7f736ee00640 1 -- start start 2024-04-28T12:51:31.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f736ee00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108f90 0x7f736819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:31.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f736ee00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f736810a180 0x7f736819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:31.900 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f736ee00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736810ab40 0x7f73681a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f736ee00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7368114b20 con 0x7f736810a180 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f736ee00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73681149a0 con 0x7f7368108f90 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f736ee00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7368114ca0 con 0x7f736810ab40 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f735f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f736810a180 0x7f736819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f7367e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108f90 0x7f736819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f736ce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736810ab40 0x7f73681a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f7367e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108f90 0x7f736819f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44606/0 (socket says 172.21.15.106:44606) 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f7367e00640 1 -- 172.21.15.106:0/2401986436 learned_addr learned my addr 172.21.15.106:0/2401986436 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f736ce00640 1 -- 172.21.15.106:0/2401986436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108f90 msgr2=0x7f736819f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.903+0000 7f736ce00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108f90 0x7f736819f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.904+0000 7f736ce00640 1 -- 172.21.15.106:0/2401986436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f736810a180 msgr2=0x7f736819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.904+0000 7f736ce00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f736810a180 0x7f736819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:31.901 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.904+0000 7f736ce00640 1 -- 172.21.15.106:0/2401986436 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73681a6da0 con 0x7f736810ab40 2024-04-28T12:51:31.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.904+0000 7f736ce00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736810ab40 0x7f73681a6730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7360009fd0 tx=0x7f736000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:31.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.904+0000 7f7365600640 1 -- 172.21.15.106:0/2401986436 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f736000ce20 con 0x7f736810ab40 2024-04-28T12:51:31.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.904+0000 7f7365600640 1 -- 172.21.15.106:0/2401986436 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7360005020 con 0x7f736810ab40 2024-04-28T12:51:31.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.904+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73681a7090 con 0x7f736810ab40 2024-04-28T12:51:31.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.904+0000 7f7365600640 1 -- 172.21.15.106:0/2401986436 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360010690 con 0x7f736810ab40 2024-04-28T12:51:31.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.904+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73681a7560 con 0x7f736810ab40 2024-04-28T12:51:31.903 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.905+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73681102b0 con 0x7f736810ab40 2024-04-28T12:51:31.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.908+0000 7f7365600640 1 -- 172.21.15.106:0/2401986436 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f73600054d0 con 0x7f736810ab40 2024-04-28T12:51:31.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.909+0000 7f7365600640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f734c0a3c40 0x7f734c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:31.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.909+0000 7f7365600640 1 -- 172.21.15.106:0/2401986436 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f73600d1060 con 0x7f736810ab40 2024-04-28T12:51:31.906 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.909+0000 7f7367e00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f734c0a3c40 0x7f734c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:31.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.910+0000 7f7367e00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f734c0a3c40 0x7f734c0a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f73580058c0 tx=0x7f7358005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:31.910 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:31.913+0000 7f7365600640 1 -- 172.21.15.106:0/2401986436 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f736009d4d0 con 0x7f736810ab40 2024-04-28T12:51:32.113 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.116+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7368063530 con 0x7f736810ab40 2024-04-28T12:51:32.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.119+0000 7f7365600640 1 -- 172.21.15.106:0/2401986436 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24928 (secure 0 0 0) 0x7f736009cdd0 con 0x7f736810ab40 2024-04-28T12:51:32.116 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:32.116 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":93,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:50:58.139109+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:51:32.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f734c0a3c40 msgr2=0x7f734c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f734c0a3c40 0x7f734c0a6100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f73580058c0 tx=0x7f7358005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:32.118 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736810ab40 msgr2=0x7f73681a6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736810ab40 0x7f73681a6730 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7360009fd0 tx=0x7f736000ef10 comp rx=0 tx=0).stop 2024-04-28T12:51:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 shutdown_connections 2024-04-28T12:51:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f734c0a3c40 0x7f734c0a6100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f736810a180 0x7f736819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736810ab40 0x7f73681a6730 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 --2- 172.21.15.106:0/2401986436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7368108f90 0x7f736819f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 >> 172.21.15.106:0/2401986436 conn(0x7f7368077f40 msgr2=0x7f736810d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 shutdown_connections 2024-04-28T12:51:32.119 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.121+0000 7f736ee00640 1 -- 172.21.15.106:0/2401986436 wait complete. 2024-04-28T12:51:32.139 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:48:52.916511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:48:53.528808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:48:57.154563+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:51:32.139 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:51:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.348+0000 7efd4d200640 1 -- 172.21.15.106:0/818041269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 msgr2=0x7efd481093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.348+0000 7efd4d200640 1 --2- 172.21.15.106:0/818041269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 0x7efd481093b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd38009860 tx=0x7efd3802a340 comp rx=0 tx=0).stop 2024-04-28T12:51:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.349+0000 7efd4d200640 1 -- 172.21.15.106:0/818041269 shutdown_connections 2024-04-28T12:51:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.349+0000 7efd4d200640 1 --2- 172.21.15.106:0/818041269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd4810ab40 0x7efd481113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.349+0000 7efd4d200640 1 --2- 172.21.15.106:0/818041269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd4810a180 0x7efd4810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.349+0000 7efd4d200640 1 --2- 172.21.15.106:0/818041269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 0x7efd481093b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.349+0000 7efd4d200640 1 -- 172.21.15.106:0/818041269 >> 172.21.15.106:0/818041269 conn(0x7efd48077f30 msgr2=0x7efd4807a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.349+0000 7efd4d200640 1 -- 172.21.15.106:0/818041269 shutdown_connections 2024-04-28T12:51:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.349+0000 7efd4d200640 1 -- 172.21.15.106:0/818041269 wait complete. 2024-04-28T12:51:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.350+0000 7efd4d200640 1 Processor -- start 2024-04-28T12:51:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.350+0000 7efd4d200640 1 -- start start 2024-04-28T12:51:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.350+0000 7efd4d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 0x7efd4819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.350+0000 7efd4d200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd4810a180 0x7efd4819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.350+0000 7efd4d200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd4810ab40 0x7efd481a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd4d200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd48114ad0 con 0x7efd4810ab40 2024-04-28T12:51:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd4d200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efd48114950 con 0x7efd4810a180 2024-04-28T12:51:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd4d200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efd48114c50 con 0x7efd48108f90 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd45e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 0x7efd4819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd45e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 0x7efd4819f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49574/0 (socket says 172.21.15.106:49574) 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd45e00640 1 -- 172.21.15.106:0/258488752 learned_addr learned my addr 172.21.15.106:0/258488752 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd46800640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd4810ab40 0x7efd481a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd45400640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd4810a180 0x7efd4819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd45e00640 1 -- 172.21.15.106:0/258488752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd4810a180 msgr2=0x7efd4819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd45e00640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd4810a180 0x7efd4819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd45e00640 1 -- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd4810ab40 msgr2=0x7efd481a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd45e00640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd4810ab40 0x7efd481a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd45e00640 1 -- 172.21.15.106:0/258488752 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd481a6e60 con 0x7efd48108f90 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.351+0000 7efd46800640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd4810ab40 0x7efd481a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.352+0000 7efd45e00640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 0x7efd4819f240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd380057c0 tx=0x7efd38032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.352+0000 7efd3ea00640 1 -- 172.21.15.106:0/258488752 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38037040 con 0x7efd48108f90 2024-04-28T12:51:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.352+0000 7efd3ea00640 1 -- 172.21.15.106:0/258488752 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd38030bd0 con 0x7efd48108f90 2024-04-28T12:51:32.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.352+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd481a70f0 con 0x7efd48108f90 2024-04-28T12:51:32.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.352+0000 7efd3ea00640 1 -- 172.21.15.106:0/258488752 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38005280 con 0x7efd48108f90 2024-04-28T12:51:32.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.352+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd481a75d0 con 0x7efd48108f90 2024-04-28T12:51:32.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.353+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd481103d0 con 0x7efd48108f90 2024-04-28T12:51:32.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.355+0000 7efd3ea00640 1 -- 172.21.15.106:0/258488752 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7efd38043020 con 0x7efd48108f90 2024-04-28T12:51:32.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.356+0000 7efd3ea00640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd2c0a3cc0 0x7efd2c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:32.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.357+0000 7efd3ea00640 1 -- 172.21.15.106:0/258488752 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7efd380efdc0 con 0x7efd48108f90 2024-04-28T12:51:32.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.357+0000 7efd45400640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd2c0a3cc0 0x7efd2c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:32.354 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.357+0000 7efd45400640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd2c0a3cc0 0x7efd2c0a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efd481a0750 tx=0x7efd34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:32.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.361+0000 7efd3ea00640 1 -- 172.21.15.106:0/258488752 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7efd380bc340 con 0x7efd48108f90 2024-04-28T12:51:32.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:32 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2401986436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:32.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:32 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2401986436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:32.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.562+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd48063530 con 0x7efd48108f90 2024-04-28T12:51:32.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.563+0000 7efd3ea00640 1 -- 172.21.15.106:0/258488752 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7efd380bbc40 con 0x7efd48108f90 2024-04-28T12:51:32.560 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T12:51:32.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd2c0a3cc0 msgr2=0x7efd2c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd2c0a3cc0 0x7efd2c0a6180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efd481a0750 tx=0x7efd34005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:32.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 msgr2=0x7efd4819f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.562 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 0x7efd4819f240 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd380057c0 tx=0x7efd38032da0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 shutdown_connections 2024-04-28T12:51:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7efd2c0a3cc0 0x7efd2c0a6180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd4810ab40 0x7efd481a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efd4810a180 0x7efd4819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 --2- 172.21.15.106:0/258488752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd48108f90 0x7efd4819f240 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.565+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 >> 172.21.15.106:0/258488752 conn(0x7efd48077f30 msgr2=0x7efd4810d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.566+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 shutdown_connections 2024-04-28T12:51:32.563 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.566+0000 7efd4d200640 1 -- 172.21.15.106:0/258488752 wait complete. 2024-04-28T12:51:32.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:32 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2401986436' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:32.581 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:51:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.787+0000 7f6721400640 1 -- 172.21.15.106:0/119880573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 msgr2=0x7f671c104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.787+0000 7f6721400640 1 --2- 172.21.15.106:0/119880573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 0x7f671c104e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f670c009900 tx=0x7f670c02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.788+0000 7f6721400640 1 -- 172.21.15.106:0/119880573 shutdown_connections 2024-04-28T12:51:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.788+0000 7f6721400640 1 --2- 172.21.15.106:0/119880573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c105c60 0x7f671c1060e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.788+0000 7f6721400640 1 --2- 172.21.15.106:0/119880573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f671c106620 0x7f671c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.788+0000 7f6721400640 1 --2- 172.21.15.106:0/119880573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 0x7f671c104e90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.788+0000 7f6721400640 1 -- 172.21.15.106:0/119880573 >> 172.21.15.106:0/119880573 conn(0x7f671c0fffe0 msgr2=0x7f671c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.788+0000 7f6721400640 1 -- 172.21.15.106:0/119880573 shutdown_connections 2024-04-28T12:51:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.788+0000 7f6721400640 1 -- 172.21.15.106:0/119880573 wait complete. 2024-04-28T12:51:32.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f6721400640 1 Processor -- start 2024-04-28T12:51:32.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f6721400640 1 -- start start 2024-04-28T12:51:32.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f6721400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 0x7f671c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:32.788 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f6721400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c105c60 0x7f671c19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f6721400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f671c106620 0x7f671c1a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f6721400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f671c1105f0 con 0x7f671c105c60 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f6721400640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f671c110470 con 0x7f671c106620 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f6721400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f671c110770 con 0x7f671c104a70 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f671be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 0x7f671c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f671be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 0x7f671c19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49592/0 (socket says 172.21.15.106:49592) 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f671be00640 1 -- 172.21.15.106:0/1020869830 learned_addr learned my addr 172.21.15.106:0/1020869830 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.789+0000 7f671b400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c105c60 0x7f671c19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.790+0000 7f6720a00640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f671c106620 0x7f671c1a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.790+0000 7f671be00640 1 -- 172.21.15.106:0/1020869830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f671c106620 msgr2=0x7f671c1a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.790+0000 7f671be00640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f671c106620 0x7f671c1a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.790+0000 7f671be00640 1 -- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c105c60 msgr2=0x7f671c19b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.790+0000 7f671be00640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c105c60 0x7f671c19b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.790+0000 7f671be00640 1 -- 172.21.15.106:0/1020869830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c1a28d0 con 0x7f671c104a70 2024-04-28T12:51:32.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.790+0000 7f671b400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c105c60 0x7f671c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:32.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.790+0000 7f671be00640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 0x7f671c19ac90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f670c031500 tx=0x7f670c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:32.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.791+0000 7f6718c00640 1 -- 172.21.15.106:0/1020869830 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c03f070 con 0x7f671c104a70 2024-04-28T12:51:32.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.791+0000 7f6718c00640 1 -- 172.21.15.106:0/1020869830 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f670c031c50 con 0x7f671c104a70 2024-04-28T12:51:32.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.791+0000 7f6718c00640 1 -- 172.21.15.106:0/1020869830 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c035030 con 0x7f671c104a70 2024-04-28T12:51:32.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.791+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c1a2b60 con 0x7f671c104a70 2024-04-28T12:51:32.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.791+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c1a3000 con 0x7f671c104a70 2024-04-28T12:51:32.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.792+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671c10be10 con 0x7f671c104a70 2024-04-28T12:51:32.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.794+0000 7f6718c00640 1 -- 172.21.15.106:0/1020869830 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f670c042070 con 0x7f671c104a70 2024-04-28T12:51:32.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.795+0000 7f6718c00640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67000a3cc0 0x7f67000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:32.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.796+0000 7f6718c00640 1 -- 172.21.15.106:0/1020869830 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f670c0f16b0 con 0x7f671c104a70 2024-04-28T12:51:32.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.796+0000 7f671b400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67000a3cc0 0x7f67000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:32.793 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.796+0000 7f671b400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67000a3cc0 0x7f67000a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f671c19c190 tx=0x7f670800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:32.794 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.798+0000 7f6718c00640 1 -- 172.21.15.106:0/1020869830 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f670c0bd4d0 con 0x7f671c104a70 2024-04-28T12:51:32.990 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.993+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f671c02dd20 con 0x7f671c104a70 2024-04-28T12:51:32.991 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.994+0000 7f6718c00640 1 -- 172.21.15.106:0/1020869830 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f670c0c17d0 con 0x7f671c104a70 2024-04-28T12:51:32.991 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:51:32.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67000a3cc0 msgr2=0x7f67000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67000a3cc0 0x7f67000a6180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f671c19c190 tx=0x7f670800b390 comp rx=0 tx=0).stop 2024-04-28T12:51:32.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 msgr2=0x7f671c19ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:32.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 0x7f671c19ac90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f670c031500 tx=0x7f670c003c10 comp rx=0 tx=0).stop 2024-04-28T12:51:32.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 shutdown_connections 2024-04-28T12:51:32.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67000a3cc0 0x7f67000a6180 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f671c105c60 0x7f671c19b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f671c106620 0x7f671c1a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 --2- 172.21.15.106:0/1020869830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f671c104a70 0x7f671c19ac90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:32.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 >> 172.21.15.106:0/1020869830 conn(0x7f671c0fffe0 msgr2=0x7f671c108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:32.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 shutdown_connections 2024-04-28T12:51:32.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:32.997+0000 7f6721400640 1 -- 172.21.15.106:0/1020869830 wait complete. 2024-04-28T12:51:33.013 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:51:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.216+0000 7fe436c00640 1 -- 172.21.15.106:0/4275571078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe430108f90 msgr2=0x7fe4301093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.216+0000 7fe436c00640 1 --2- 172.21.15.106:0/4275571078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe430108f90 0x7fe4301093b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe420009900 tx=0x7fe42002a410 comp rx=0 tx=0).stop 2024-04-28T12:51:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.216+0000 7fe436c00640 1 -- 172.21.15.106:0/4275571078 shutdown_connections 2024-04-28T12:51:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.216+0000 7fe436c00640 1 --2- 172.21.15.106:0/4275571078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe43010ab40 0x7fe4301113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.216+0000 7fe436c00640 1 --2- 172.21.15.106:0/4275571078 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe43010a180 0x7fe43010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.216+0000 7fe436c00640 1 --2- 172.21.15.106:0/4275571078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe430108f90 0x7fe4301093b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.216+0000 7fe436c00640 1 -- 172.21.15.106:0/4275571078 >> 172.21.15.106:0/4275571078 conn(0x7fe430077f30 msgr2=0x7fe43007a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.216+0000 7fe436c00640 1 -- 172.21.15.106:0/4275571078 shutdown_connections 2024-04-28T12:51:33.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.217+0000 7fe436c00640 1 -- 172.21.15.106:0/4275571078 wait complete. 2024-04-28T12:51:33.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.217+0000 7fe436c00640 1 Processor -- start 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.217+0000 7fe436c00640 1 -- start start 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe436c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe430108f90 0x7fe43019f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe436c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe43010a180 0x7fe43019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe436c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe43010ab40 0x7fe4301a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe436c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe430114b20 con 0x7fe43010a180 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe436c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4301149a0 con 0x7fe43010ab40 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe436c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe430114ca0 con 0x7fe430108f90 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe42f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe43010a180 0x7fe43019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe42f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe43010a180 0x7fe43019f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45062/0 (socket says 172.21.15.106:45062) 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe42f400640 1 -- 172.21.15.106:0/4160823138 learned_addr learned my addr 172.21.15.106:0/4160823138 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe42fe00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe430108f90 0x7fe43019f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:33.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.218+0000 7fe434c00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe43010ab40 0x7fe4301a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe42f400640 1 -- 172.21.15.106:0/4160823138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe430108f90 msgr2=0x7fe43019f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe42f400640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe430108f90 0x7fe43019f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe42f400640 1 -- 172.21.15.106:0/4160823138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe43010ab40 msgr2=0x7fe4301a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe42f400640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe43010ab40 0x7fe4301a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe42f400640 1 -- 172.21.15.106:0/4160823138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4301a6dc0 con 0x7fe43010a180 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe42f400640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe43010a180 0x7fe43019f6e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00e9f0 tx=0x7fe41c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe42cc00640 1 -- 172.21.15.106:0/4160823138 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c00cd80 con 0x7fe43010a180 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe42cc00640 1 -- 172.21.15.106:0/4160823138 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe41c0054d0 con 0x7fe43010a180 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4301a70b0 con 0x7fe43010a180 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe42cc00640 1 -- 172.21.15.106:0/4160823138 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c010690 con 0x7fe43010a180 2024-04-28T12:51:33.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.219+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4301a75b0 con 0x7fe43010a180 2024-04-28T12:51:33.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.221+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4301102b0 con 0x7fe43010a180 2024-04-28T12:51:33.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.223+0000 7fe42cc00640 1 -- 172.21.15.106:0/4160823138 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe41c004e90 con 0x7fe43010a180 2024-04-28T12:51:33.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.224+0000 7fe42cc00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe4140a3c40 0x7fe4140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:33.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.225+0000 7fe42cc00640 1 -- 172.21.15.106:0/4160823138 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fe41c0d0d90 con 0x7fe43010a180 2024-04-28T12:51:33.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.225+0000 7fe42fe00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe4140a3c40 0x7fe4140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:33.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.225+0000 7fe42fe00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe4140a3c40 0x7fe4140a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe420005ab0 tx=0x7fe420005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.229+0000 7fe42cc00640 1 -- 172.21.15.106:0/4160823138 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe41c09d200 con 0x7fe43010a180 2024-04-28T12:51:33.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:33 smithi106 ceph-mon[28081]: pgmap v401: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:51:33.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:33 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/258488752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:33.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:33 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1020869830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:33.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.429+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe43002dd20 con 0x7fe43010a180 2024-04-28T12:51:33.426 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.430+0000 7fe42cc00640 1 -- 172.21.15.106:0/4160823138 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fe41c09cb00 con 0x7fe43010a180 2024-04-28T12:51:33.427 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.432+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe4140a3c40 msgr2=0x7fe4140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.432+0000 7fe436c00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe4140a3c40 0x7fe4140a6100 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe420005ab0 tx=0x7fe420005a20 comp rx=0 tx=0).stop 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.432+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe43010a180 msgr2=0x7fe43019f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.432+0000 7fe436c00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe43010a180 0x7fe43019f6e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00e9f0 tx=0x7fe41c00edf0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.433+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 shutdown_connections 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.433+0000 7fe436c00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe4140a3c40 0x7fe4140a6100 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.433+0000 7fe436c00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe43010a180 0x7fe43019f6e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.433+0000 7fe436c00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe43010ab40 0x7fe4301a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.433+0000 7fe436c00640 1 --2- 172.21.15.106:0/4160823138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe430108f90 0x7fe43019f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.433+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 >> 172.21.15.106:0/4160823138 conn(0x7fe430077f30 msgr2=0x7fe43010d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.433+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 shutdown_connections 2024-04-28T12:51:33.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.433+0000 7fe436c00640 1 -- 172.21.15.106:0/4160823138 wait complete. 2024-04-28T12:51:33.448 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:51:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:33 smithi171 ceph-mon[63127]: pgmap v401: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:51:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:33 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/258488752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:33.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:33 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1020869830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:33 smithi129 ceph-mon[62418]: pgmap v401: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:51:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:33 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/258488752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:33.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:33 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1020869830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.645+0000 7f6127200640 1 -- 172.21.15.106:0/3698116434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6128105c60 msgr2=0x7f61281060e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.645+0000 7f6127200640 1 --2- 172.21.15.106:0/3698116434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6128105c60 0x7f61281060e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6114009860 tx=0x7f611402a340 comp rx=0 tx=0).stop 2024-04-28T12:51:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.645+0000 7f6127200640 1 -- 172.21.15.106:0/3698116434 shutdown_connections 2024-04-28T12:51:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.645+0000 7f6127200640 1 --2- 172.21.15.106:0/3698116434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6128106620 0x7f612810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.645+0000 7f6127200640 1 --2- 172.21.15.106:0/3698116434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6128105c60 0x7f61281060e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.645+0000 7f6127200640 1 --2- 172.21.15.106:0/3698116434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6128104a70 0x7f6128104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.645+0000 7f6127200640 1 -- 172.21.15.106:0/3698116434 >> 172.21.15.106:0/3698116434 conn(0x7f61280fffe0 msgr2=0x7f6128102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:33.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.645+0000 7f6127200640 1 -- 172.21.15.106:0/3698116434 shutdown_connections 2024-04-28T12:51:33.643 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.645+0000 7f6127200640 1 -- 172.21.15.106:0/3698116434 wait complete. 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.646+0000 7f6127200640 1 Processor -- start 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.646+0000 7f6127200640 1 -- start start 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6127200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6128104a70 0x7f612819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6127200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6128105c60 0x7f612819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6127200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6128106620 0x7f61281a2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6127200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f61281105a0 con 0x7f6128105c60 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6127200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6128110420 con 0x7f6128104a70 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6127200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6128110720 con 0x7f6128106620 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6125400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6128105c60 0x7f612819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6125e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6128104a70 0x7f612819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:33.644 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6125400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6128105c60 0x7f612819b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45076/0 (socket says 172.21.15.106:45076) 2024-04-28T12:51:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6125400640 1 -- 172.21.15.106:0/774170786 learned_addr learned my addr 172.21.15.106:0/774170786 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:33.645 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6126800640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6128106620 0x7f61281a2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6125400640 1 -- 172.21.15.106:0/774170786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6128106620 msgr2=0x7f61281a2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6125400640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6128106620 0x7f61281a2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6125400640 1 -- 172.21.15.106:0/774170786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6128104a70 msgr2=0x7f612819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6125400640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6128104a70 0x7f612819acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.647+0000 7f6125400640 1 -- 172.21.15.106:0/774170786 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61281a2900 con 0x7f6128105c60 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.648+0000 7f6125400640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6128105c60 0x7f612819b230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6114005940 tx=0x7f611402ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.648+0000 7f611ea00640 1 -- 172.21.15.106:0/774170786 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114040020 con 0x7f6128105c60 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.648+0000 7f611ea00640 1 -- 172.21.15.106:0/774170786 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6114030e50 con 0x7f6128105c60 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.648+0000 7f611ea00640 1 -- 172.21.15.106:0/774170786 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114037040 con 0x7f6128105c60 2024-04-28T12:51:33.646 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.648+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61281a2b90 con 0x7f6128105c60 2024-04-28T12:51:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.648+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61281a30d0 con 0x7f6128105c60 2024-04-28T12:51:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.649+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612810be10 con 0x7f6128105c60 2024-04-28T12:51:33.647 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.651+0000 7f611ea00640 1 -- 172.21.15.106:0/774170786 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6114043020 con 0x7f6128105c60 2024-04-28T12:51:33.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.652+0000 7f611ea00640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f610c0a3cc0 0x7f610c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:33.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.652+0000 7f611ea00640 1 -- 172.21.15.106:0/774170786 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f61140f05c0 con 0x7f6128105c60 2024-04-28T12:51:33.649 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.652+0000 7f6125e00640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f610c0a3cc0 0x7f610c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:33.650 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.653+0000 7f6125e00640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f610c0a3cc0 0x7f610c0a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f612819c1c0 tx=0x7f6118005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:33.653 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.656+0000 7f611ea00640 1 -- 172.21.15.106:0/774170786 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f611404a020 con 0x7f6128105c60 2024-04-28T12:51:33.855 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.858+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f612802dcf0 con 0x7f6128105c60 2024-04-28T12:51:33.856 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.859+0000 7f611ea00640 1 -- 172.21.15.106:0/774170786 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f61140bcb40 con 0x7f6128105c60 2024-04-28T12:51:33.856 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:51:33.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f610c0a3cc0 msgr2=0x7f610c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f610c0a3cc0 0x7f610c0a6180 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f612819c1c0 tx=0x7f6118005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:33.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6128105c60 msgr2=0x7f612819b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:33.859 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6128105c60 0x7f612819b230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6114005940 tx=0x7f611402ae20 comp rx=0 tx=0).stop 2024-04-28T12:51:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 shutdown_connections 2024-04-28T12:51:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f610c0a3cc0 0x7f610c0a6180 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6128105c60 0x7f612819b230 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6128106620 0x7f61281a2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 --2- 172.21.15.106:0/774170786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6128104a70 0x7f612819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 >> 172.21.15.106:0/774170786 conn(0x7f61280fffe0 msgr2=0x7f6128108fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 shutdown_connections 2024-04-28T12:51:33.860 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:33.862+0000 7f6127200640 1 -- 172.21.15.106:0/774170786 wait complete. 2024-04-28T12:51:33.878 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:51:34.076 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.079+0000 7f4823e00640 1 -- 172.21.15.106:0/2238512690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4824075390 msgr2=0x7f4824075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.079+0000 7f4823e00640 1 --2- 172.21.15.106:0/2238512690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4824075390 0x7f4824075810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4810009930 tx=0x7f481002a420 comp rx=0 tx=0).stop 2024-04-28T12:51:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.079+0000 7f4823e00640 1 -- 172.21.15.106:0/2238512690 shutdown_connections 2024-04-28T12:51:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.079+0000 7f4823e00640 1 --2- 172.21.15.106:0/2238512690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4824075390 0x7f4824075810 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.079+0000 7f4823e00640 1 --2- 172.21.15.106:0/2238512690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f482410ed20 0x7f4824111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.079+0000 7f4823e00640 1 --2- 172.21.15.106:0/2238512690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4824076970 0x7f4824074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.079+0000 7f4823e00640 1 -- 172.21.15.106:0/2238512690 >> 172.21.15.106:0/2238512690 conn(0x7f48240fdcc0 msgr2=0x7f4824100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.079+0000 7f4823e00640 1 -- 172.21.15.106:0/2238512690 shutdown_connections 2024-04-28T12:51:34.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.079+0000 7f4823e00640 1 -- 172.21.15.106:0/2238512690 wait complete. 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.081+0000 7f4823e00640 1 Processor -- start 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.081+0000 7f4823e00640 1 -- start start 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.081+0000 7f4823e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4824075390 0x7f482419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.081+0000 7f4823e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4824076970 0x7f482419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.081+0000 7f4823e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f482410ed20 0x7f48241a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.081+0000 7f4823e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4824114870 con 0x7f482410ed20 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.081+0000 7f4823e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48241146f0 con 0x7f4824076970 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.081+0000 7f4823e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f48241149f0 con 0x7f4824075390 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4824075390 0x7f482419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4824075390 0x7f482419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49642/0 (socket says 172.21.15.106:49642) 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822a00640 1 -- 172.21.15.106:0/4127164627 learned_addr learned my addr 172.21.15.106:0/4127164627 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4823400640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f482410ed20 0x7f48241a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822000640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4824076970 0x7f482419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:34.079 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822a00640 1 -- 172.21.15.106:0/4127164627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4824076970 msgr2=0x7f482419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822a00640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4824076970 0x7f482419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822a00640 1 -- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f482410ed20 msgr2=0x7f48241a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822a00640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f482410ed20 0x7f48241a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.080 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822a00640 1 -- 172.21.15.106:0/4127164627 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48241a6bb0 con 0x7f4824075390 2024-04-28T12:51:34.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.082+0000 7f4822a00640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4824075390 0x7f482419ef90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f481400ea60 tx=0x7f481400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:34.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.083+0000 7f481b400640 1 -- 172.21.15.106:0/4127164627 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481400cde0 con 0x7f4824075390 2024-04-28T12:51:34.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.083+0000 7f481b400640 1 -- 172.21.15.106:0/4127164627 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48140035f0 con 0x7f4824075390 2024-04-28T12:51:34.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.083+0000 7f481b400640 1 -- 172.21.15.106:0/4127164627 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4814010690 con 0x7f4824075390 2024-04-28T12:51:34.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.083+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48241a6ea0 con 0x7f4824075390 2024-04-28T12:51:34.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.083+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48241a7250 con 0x7f4824075390 2024-04-28T12:51:34.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.084+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4824104ef0 con 0x7f4824075390 2024-04-28T12:51:34.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.086+0000 7f481b400640 1 -- 172.21.15.106:0/4127164627 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f48140037a0 con 0x7f4824075390 2024-04-28T12:51:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.087+0000 7f481b400640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f48080a3cc0 0x7f48080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:34.084 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.088+0000 7f481b400640 1 -- 172.21.15.106:0/4127164627 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f48140d0be0 con 0x7f4824075390 2024-04-28T12:51:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.088+0000 7f4822000640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f48080a3cc0 0x7f48080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:34.085 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.088+0000 7f4822000640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f48080a3cc0 0x7f48080a6180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4810006010 tx=0x7f48100314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:34.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.092+0000 7f481b400640 1 -- 172.21.15.106:0/4127164627 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f481409d050 con 0x7f4824075390 2024-04-28T12:51:34.341 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.344+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f482402dad0 con 0x7f4824075390 2024-04-28T12:51:34.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.345+0000 7f481b400640 1 -- 172.21.15.106:0/4127164627 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11953 (secure 0 0 0) 0x7f481409c950 con 0x7f4824075390 2024-04-28T12:51:34.343 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:34.343 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:49:30.023676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24548,"mds_3":24538,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24548,24520,24578,24538,24566]},"id":1}]} 2024-04-28T12:51:34.343 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-28T12:51:34.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f48080a3cc0 msgr2=0x7f48080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f48080a3cc0 0x7f48080a6180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4810006010 tx=0x7f48100314a0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4824075390 msgr2=0x7f482419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4824075390 0x7f482419ef90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f481400ea60 tx=0x7f481400ef20 comp rx=0 tx=0).stop 2024-04-28T12:51:34.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 shutdown_connections 2024-04-28T12:51:34.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f48080a3cc0 0x7f48080a6180 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f482410ed20 0x7f48241a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4824076970 0x7f482419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 --2- 172.21.15.106:0/4127164627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4824075390 0x7f482419ef90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 >> 172.21.15.106:0/4127164627 conn(0x7f48240fdcc0 msgr2=0x7f48241020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:34.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.348+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 shutdown_connections 2024-04-28T12:51:34.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.349+0000 7f4823e00640 1 -- 172.21.15.106:0/4127164627 wait complete. 2024-04-28T12:51:34.364 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:51:34.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:34 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/4160823138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:34.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:34 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/774170786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:34.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:34 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/4160823138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:34.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:34 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/774170786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:34.568 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.571+0000 7f673b200640 1 -- 172.21.15.106:0/495802848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f673c109f80 msgr2=0x7f673c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.571+0000 7f673b200640 1 --2- 172.21.15.106:0/495802848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f673c109f80 0x7f673c10a400 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6724009900 tx=0x7f672402a3f0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.571+0000 7f673b200640 1 -- 172.21.15.106:0/495802848 shutdown_connections 2024-04-28T12:51:34.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.571+0000 7f673b200640 1 --2- 172.21.15.106:0/495802848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f673c10a940 0x7f673c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.571+0000 7f673b200640 1 --2- 172.21.15.106:0/495802848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f673c109f80 0x7f673c10a400 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.571+0000 7f673b200640 1 --2- 172.21.15.106:0/495802848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f673c108d90 0x7f673c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.571+0000 7f673b200640 1 -- 172.21.15.106:0/495802848 >> 172.21.15.106:0/495802848 conn(0x7f673c077d30 msgr2=0x7f673c07a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:34.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.571+0000 7f673b200640 1 -- 172.21.15.106:0/495802848 shutdown_connections 2024-04-28T12:51:34.569 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.571+0000 7f673b200640 1 -- 172.21.15.106:0/495802848 wait complete. 2024-04-28T12:51:34.571 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.572+0000 7f673b200640 1 Processor -- start 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f673b200640 1 -- start start 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f673b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f673c108d90 0x7f673c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f673b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f673c109f80 0x7f673c19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f673b200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f673c10a940 0x7f673c1a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f673b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f673c1148a0 con 0x7f673c109f80 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f673b200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f673c114720 con 0x7f673c10a940 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f673b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f673c114a20 con 0x7f673c108d90 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f6739400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f673c109f80 0x7f673c19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f6739e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f673c108d90 0x7f673c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f6739400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f673c109f80 0x7f673c19f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45122/0 (socket says 172.21.15.106:45122) 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f6739400640 1 -- 172.21.15.106:0/4096984745 learned_addr learned my addr 172.21.15.106:0/4096984745 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:34.572 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f6739e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f673c108d90 0x7f673c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49654/0 (socket says 172.21.15.106:49654) 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.573+0000 7f673a800640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f673c10a940 0x7f673c1a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f6739e00640 1 -- 172.21.15.106:0/4096984745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f673c10a940 msgr2=0x7f673c1a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f6739e00640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f673c10a940 0x7f673c1a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f6739e00640 1 -- 172.21.15.106:0/4096984745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f673c109f80 msgr2=0x7f673c19f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f6739e00640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f673c109f80 0x7f673c19f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f6739e00640 1 -- 172.21.15.106:0/4096984745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673c1a6bf0 con 0x7f673c108d90 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f6739e00640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f673c108d90 0x7f673c19efd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f672c00e9c0 tx=0x7f672c00edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f6732a00640 1 -- 172.21.15.106:0/4096984745 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c00ccc0 con 0x7f673c108d90 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f6732a00640 1 -- 172.21.15.106:0/4096984745 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f672c0108e0 con 0x7f673c108d90 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f6732a00640 1 -- 172.21.15.106:0/4096984745 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c018840 con 0x7f673c108d90 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.574+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f673c1a6ee0 con 0x7f673c108d90 2024-04-28T12:51:34.573 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.575+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f673c1a72f0 con 0x7f673c108d90 2024-04-28T12:51:34.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.576+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673c1101a0 con 0x7f673c108d90 2024-04-28T12:51:34.574 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.578+0000 7f6732a00640 1 -- 172.21.15.106:0/4096984745 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f672c01f020 con 0x7f673c108d90 2024-04-28T12:51:34.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:34 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4160823138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:34.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:34 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/774170786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:34.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.579+0000 7f6732a00640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67200a3c40 0x7f67200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:34.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.579+0000 7f6732a00640 1 -- 172.21.15.106:0/4096984745 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f672c0d08c0 con 0x7f673c108d90 2024-04-28T12:51:34.576 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.579+0000 7f6739400640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67200a3c40 0x7f67200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:34.577 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.580+0000 7f6739400640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67200a3c40 0x7f67200a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6724009230 tx=0x7f67240058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:34.580 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.583+0000 7f6732a00640 1 -- 172.21.15.106:0/4096984745 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f672c09cd30 con 0x7f673c108d90 2024-04-28T12:51:34.781 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.784+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f673c02dd50 con 0x7f673c108d90 2024-04-28T12:51:34.784 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.787+0000 7f6732a00640 1 -- 172.21.15.106:0/4096984745 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24928 (secure 0 0 0) 0x7f672c09c630 con 0x7f673c108d90 2024-04-28T12:51:34.784 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:34.784 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":93,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:50:58.139109+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:51:34.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67200a3c40 msgr2=0x7f67200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.786 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67200a3c40 0x7f67200a6100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6724009230 tx=0x7f67240058d0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f673c108d90 msgr2=0x7f673c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f673c108d90 0x7f673c19efd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f672c00e9c0 tx=0x7f672c00edc0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 shutdown_connections 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f67200a3c40 0x7f67200a6100 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f673c109f80 0x7f673c19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f673c10a940 0x7f673c1a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 --2- 172.21.15.106:0/4096984745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f673c108d90 0x7f673c19efd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 >> 172.21.15.106:0/4096984745 conn(0x7f673c077d30 msgr2=0x7f673c10d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 shutdown_connections 2024-04-28T12:51:34.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:34.789+0000 7f673b200640 1 -- 172.21.15.106:0/4096984745 wait complete. 2024-04-28T12:51:34.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:48:52.916511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:48:53.528808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:48:57.154563+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T12:51:34.806 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T12:51:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.023+0000 7fd099200640 1 -- 172.21.15.106:0/748522984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd09410ab40 msgr2=0x7fd0941113d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.023+0000 7fd099200640 1 --2- 172.21.15.106:0/748522984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd09410ab40 0x7fd0941113d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009be0 tx=0x7fd08c02a6d0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.024+0000 7fd099200640 1 -- 172.21.15.106:0/748522984 shutdown_connections 2024-04-28T12:51:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.024+0000 7fd099200640 1 --2- 172.21.15.106:0/748522984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd09410a180 0x7fd09410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.024+0000 7fd099200640 1 --2- 172.21.15.106:0/748522984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd09410ab40 0x7fd0941113d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.024+0000 7fd099200640 1 --2- 172.21.15.106:0/748522984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd094108f90 0x7fd0941093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.024+0000 7fd099200640 1 -- 172.21.15.106:0/748522984 >> 172.21.15.106:0/748522984 conn(0x7fd094077f30 msgr2=0x7fd09407a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:35.021 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.024+0000 7fd099200640 1 -- 172.21.15.106:0/748522984 shutdown_connections 2024-04-28T12:51:35.022 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.024+0000 7fd099200640 1 -- 172.21.15.106:0/748522984 wait complete. 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd099200640 1 Processor -- start 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd099200640 1 -- start start 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd099200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd094108f90 0x7fd09419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd099200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd09410a180 0x7fd09419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd099200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd09410ab40 0x7fd0941a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd099200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd094114a90 con 0x7fd094108f90 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd099200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd094114910 con 0x7fd09410a180 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd099200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd094114c10 con 0x7fd09410ab40 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd091e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd094108f90 0x7fd09419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd091e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd094108f90 0x7fd09419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45144/0 (socket says 172.21.15.106:45144) 2024-04-28T12:51:35.024 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.025+0000 7fd091e00640 1 -- 172.21.15.106:0/2017892643 learned_addr learned my addr 172.21.15.106:0/2017892643 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.026+0000 7fd092800640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd09410ab40 0x7fd0941a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.026+0000 7fd08be00640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd09410a180 0x7fd09419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.026+0000 7fd091e00640 1 -- 172.21.15.106:0/2017892643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd09410ab40 msgr2=0x7fd0941a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.026+0000 7fd091e00640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd09410ab40 0x7fd0941a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.026+0000 7fd091e00640 1 -- 172.21.15.106:0/2017892643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd09410a180 msgr2=0x7fd09419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.026+0000 7fd091e00640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd09410a180 0x7fd09419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.026+0000 7fd091e00640 1 -- 172.21.15.106:0/2017892643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0941a6db0 con 0x7fd094108f90 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.026+0000 7fd091e00640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd094108f90 0x7fd09419f190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd08400e9c0 tx=0x7fd08400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.026+0000 7fd08aa00640 1 -- 172.21.15.106:0/2017892643 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08400cd80 con 0x7fd094108f90 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.027+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0941a70a0 con 0x7fd094108f90 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.027+0000 7fd08aa00640 1 -- 172.21.15.106:0/2017892643 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0840035f0 con 0x7fd094108f90 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.027+0000 7fd08aa00640 1 -- 172.21.15.106:0/2017892643 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084010690 con 0x7fd094108f90 2024-04-28T12:51:35.025 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.027+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0941a7640 con 0x7fd094108f90 2024-04-28T12:51:35.026 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.028+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd094110300 con 0x7fd094108f90 2024-04-28T12:51:35.027 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.030+0000 7fd08aa00640 1 -- 172.21.15.106:0/2017892643 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd0840037a0 con 0x7fd094108f90 2024-04-28T12:51:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.032+0000 7fd08aa00640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0780a3c40 0x7fd0780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.032+0000 7fd08aa00640 1 -- 172.21.15.106:0/2017892643 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fd0840d0d00 con 0x7fd094108f90 2024-04-28T12:51:35.029 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.033+0000 7fd08be00640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0780a3c40 0x7fd0780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.030 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.033+0000 7fd08be00640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0780a3c40 0x7fd0780a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd0941a06a0 tx=0x7fd07c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:35.033 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.037+0000 7fd08aa00640 1 -- 172.21.15.106:0/2017892643 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd08409d170 con 0x7fd094108f90 2024-04-28T12:51:35.235 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.238+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd094063530 con 0x7fd094108f90 2024-04-28T12:51:35.236 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.239+0000 7fd08aa00640 1 -- 172.21.15.106:0/2017892643 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fd08409ca70 con 0x7fd094108f90 2024-04-28T12:51:35.236 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0780a3c40 msgr2=0x7fd0780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0780a3c40 0x7fd0780a6100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd0941a06a0 tx=0x7fd07c009260 comp rx=0 tx=0).stop 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd094108f90 msgr2=0x7fd09419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd094108f90 0x7fd09419f190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd08400e9c0 tx=0x7fd08400ee80 comp rx=0 tx=0).stop 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 shutdown_connections 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0780a3c40 0x7fd0780a6100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd09410ab40 0x7fd0941a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd09410a180 0x7fd09419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 --2- 172.21.15.106:0/2017892643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd094108f90 0x7fd09419f190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.239 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 >> 172.21.15.106:0/2017892643 conn(0x7fd094077f30 msgr2=0x7fd09410d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:35.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 shutdown_connections 2024-04-28T12:51:35.240 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.242+0000 7fd099200640 1 -- 172.21.15.106:0/2017892643 wait complete. 2024-04-28T12:51:35.258 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T12:51:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.468+0000 7fd69aa00640 1 -- 172.21.15.106:0/1997168835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694102840 msgr2=0x7fd694102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.468+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1997168835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694102840 0x7fd694102c60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd688009860 tx=0x7fd68802a340 comp rx=0 tx=0).stop 2024-04-28T12:51:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.468+0000 7fd69aa00640 1 -- 172.21.15.106:0/1997168835 shutdown_connections 2024-04-28T12:51:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.468+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1997168835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd694103a30 0x7fd694103eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.468+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1997168835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6941043f0 0x7fd69410ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.468+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1997168835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694102840 0x7fd694102c60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.468+0000 7fd69aa00640 1 -- 172.21.15.106:0/1997168835 >> 172.21.15.106:0/1997168835 conn(0x7fd6940fddd0 msgr2=0x7fd6941001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.468+0000 7fd69aa00640 1 -- 172.21.15.106:0/1997168835 shutdown_connections 2024-04-28T12:51:35.466 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.468+0000 7fd69aa00640 1 -- 172.21.15.106:0/1997168835 wait complete. 2024-04-28T12:51:35.469 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.469+0000 7fd69aa00640 1 Processor -- start 2024-04-28T12:51:35.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.469+0000 7fd69aa00640 1 -- start start 2024-04-28T12:51:35.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.469+0000 7fd69aa00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd694102840 0x7fd69419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.469+0000 7fd69aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694103a30 0x7fd69419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.469+0000 7fd69aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6941043f0 0x7fd6941a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.469+0000 7fd69aa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd69410e370 con 0x7fd6941043f0 2024-04-28T12:51:35.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.469+0000 7fd69aa00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd69410e1f0 con 0x7fd694102840 2024-04-28T12:51:35.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.469+0000 7fd69aa00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd69410e4f0 con 0x7fd694103a30 2024-04-28T12:51:35.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd69a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6941043f0 0x7fd6941a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.470 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd698c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694103a30 0x7fd69419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd699600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd694102840 0x7fd69419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd698c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694103a30 0x7fd69419b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49686/0 (socket says 172.21.15.106:49686) 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd69a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6941043f0 0x7fd6941a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45164/0 (socket says 172.21.15.106:45164) 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd698c00640 1 -- 172.21.15.106:0/1581397541 learned_addr learned my addr 172.21.15.106:0/1581397541 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd699600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd694102840 0x7fd69419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44726/0 (socket says 172.21.15.106:44726) 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd699600640 1 -- 172.21.15.106:0/1581397541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694103a30 msgr2=0x7fd69419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd699600640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694103a30 0x7fd69419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd699600640 1 -- 172.21.15.106:0/1581397541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6941043f0 msgr2=0x7fd6941a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd699600640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6941043f0 0x7fd6941a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd699600640 1 -- 172.21.15.106:0/1581397541 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6941a28c0 con 0x7fd694102840 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd698c00640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694103a30 0x7fd69419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.470+0000 7fd699600640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd694102840 0x7fd69419ac80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd68802a860 tx=0x7fd688032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.471+0000 7fd692000640 1 -- 172.21.15.106:0/1581397541 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688037040 con 0x7fd694102840 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.471+0000 7fd692000640 1 -- 172.21.15.106:0/1581397541 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd688030bd0 con 0x7fd694102840 2024-04-28T12:51:35.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.471+0000 7fd692000640 1 -- 172.21.15.106:0/1581397541 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688003850 con 0x7fd694102840 2024-04-28T12:51:35.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.471+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6941a2b50 con 0x7fd694102840 2024-04-28T12:51:35.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.471+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6941a2fe0 con 0x7fd694102840 2024-04-28T12:51:35.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.472+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6941a3210 con 0x7fd694102840 2024-04-28T12:51:35.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.474+0000 7fd692000640 1 -- 172.21.15.106:0/1581397541 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd688043020 con 0x7fd694102840 2024-04-28T12:51:35.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.475+0000 7fd692000640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd67c0a3c40 0x7fd67c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.475+0000 7fd692000640 1 -- 172.21.15.106:0/1581397541 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fd6880efb50 con 0x7fd694102840 2024-04-28T12:51:35.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.475+0000 7fd698c00640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd67c0a3c40 0x7fd67c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.476+0000 7fd698c00640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd67c0a3c40 0x7fd67c0a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd69419c180 tx=0x7fd680009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:35.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.478+0000 7fd692000640 1 -- 172.21.15.106:0/1581397541 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd68804a060 con 0x7fd694102840 2024-04-28T12:51:35.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:35 smithi171 ceph-mon[63127]: pgmap v402: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:51:35.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:35 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/4127164627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:35.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:35 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/4096984745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:35.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:35 smithi129 ceph-mon[62418]: pgmap v402: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:51:35.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:35 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/4127164627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:35.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:35 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/4096984745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:35.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:35 smithi106 ceph-mon[28081]: pgmap v402: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:51:35.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:35 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4127164627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:35.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:35 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/4096984745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:35.661 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.664+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6941a35a0 con 0x7fd694102840 2024-04-28T12:51:35.662 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.665+0000 7fd692000640 1 -- 172.21.15.106:0/1581397541 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7fd6880bc150 con 0x7fd694102840 2024-04-28T12:51:35.662 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:51:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd67c0a3c40 msgr2=0x7fd67c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd67c0a3c40 0x7fd67c0a6100 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd69419c180 tx=0x7fd680009260 comp rx=0 tx=0).stop 2024-04-28T12:51:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd694102840 msgr2=0x7fd69419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd694102840 0x7fd69419ac80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd68802a860 tx=0x7fd688032dc0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 shutdown_connections 2024-04-28T12:51:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd67c0a3c40 0x7fd67c0a6100 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6941043f0 0x7fd6941a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd694103a30 0x7fd69419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 --2- 172.21.15.106:0/1581397541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd694102840 0x7fd69419ac80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 >> 172.21.15.106:0/1581397541 conn(0x7fd6940fddd0 msgr2=0x7fd6941091d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:35.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 shutdown_connections 2024-04-28T12:51:35.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.668+0000 7fd69aa00640 1 -- 172.21.15.106:0/1581397541 wait complete. 2024-04-28T12:51:35.683 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T12:51:35.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.893+0000 7f005fe00640 1 -- 172.21.15.106:0/8689449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0060109f80 msgr2=0x7f006010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.893+0000 7f005fe00640 1 --2- 172.21.15.106:0/8689449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0060109f80 0x7f006010a400 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f004c009900 tx=0x7f004c02a400 comp rx=0 tx=0).stop 2024-04-28T12:51:35.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.893+0000 7f005fe00640 1 -- 172.21.15.106:0/8689449 shutdown_connections 2024-04-28T12:51:35.890 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.893+0000 7f005fe00640 1 --2- 172.21.15.106:0/8689449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f006010a940 0x7f00601111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.893+0000 7f005fe00640 1 --2- 172.21.15.106:0/8689449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0060109f80 0x7f006010a400 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.893+0000 7f005fe00640 1 --2- 172.21.15.106:0/8689449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0060108d90 0x7f00601091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.893+0000 7f005fe00640 1 -- 172.21.15.106:0/8689449 >> 172.21.15.106:0/8689449 conn(0x7f0060077d30 msgr2=0x7f006007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:35.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.893+0000 7f005fe00640 1 -- 172.21.15.106:0/8689449 shutdown_connections 2024-04-28T12:51:35.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.893+0000 7f005fe00640 1 -- 172.21.15.106:0/8689449 wait complete. 2024-04-28T12:51:35.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.894+0000 7f005fe00640 1 Processor -- start 2024-04-28T12:51:35.891 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005fe00640 1 -- start start 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0060108d90 0x7f006019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005fe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0060109f80 0x7f006019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f006010a940 0x7f00601a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f00601148f0 con 0x7f0060108d90 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005fe00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0060114770 con 0x7f0060109f80 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0060114a70 con 0x7f006010a940 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0060108d90 0x7f006019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0060108d90 0x7f006019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45178/0 (socket says 172.21.15.106:45178) 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005ea00640 1 -- 172.21.15.106:0/2433776931 learned_addr learned my addr 172.21.15.106:0/2433776931 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005e000640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0060109f80 0x7f006019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.895+0000 7f005f400640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f006010a940 0x7f00601a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.896+0000 7f005ea00640 1 -- 172.21.15.106:0/2433776931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f006010a940 msgr2=0x7f00601a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.893 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.896+0000 7f005ea00640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f006010a940 0x7f00601a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.896+0000 7f005ea00640 1 -- 172.21.15.106:0/2433776931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0060109f80 msgr2=0x7f006019f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:35.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.896+0000 7f005ea00640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0060109f80 0x7f006019f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:35.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.896+0000 7f005ea00640 1 -- 172.21.15.106:0/2433776931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00601a6be0 con 0x7f0060108d90 2024-04-28T12:51:35.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.896+0000 7f005e000640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0060109f80 0x7f006019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:35.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.896+0000 7f005ea00640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0060108d90 0x7f006019efc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f005000e9c0 tx=0x7f005000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:35.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.896+0000 7f0057400640 1 -- 172.21.15.106:0/2433776931 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005000cde0 con 0x7f0060108d90 2024-04-28T12:51:35.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.897+0000 7f0057400640 1 -- 172.21.15.106:0/2433776931 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00500108e0 con 0x7f0060108d90 2024-04-28T12:51:35.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.897+0000 7f0057400640 1 -- 172.21.15.106:0/2433776931 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050018960 con 0x7f0060108d90 2024-04-28T12:51:35.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.897+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00601a6ed0 con 0x7f0060108d90 2024-04-28T12:51:35.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.897+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00601a7330 con 0x7f0060108d90 2024-04-28T12:51:35.895 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.898+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00601100d0 con 0x7f0060108d90 2024-04-28T12:51:35.896 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.900+0000 7f0057400640 1 -- 172.21.15.106:0/2433776931 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f005001f020 con 0x7f0060108d90 2024-04-28T12:51:35.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.901+0000 7f0057400640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f00440a3c40 0x7f00440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:35.898 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.902+0000 7f0057400640 1 -- 172.21.15.106:0/2433776931 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f00500d0c10 con 0x7f0060108d90 2024-04-28T12:51:35.899 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.902+0000 7f005e000640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f00440a3c40 0x7f00440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:35.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.903+0000 7f005e000640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f00440a3c40 0x7f00440a6100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f004c031500 tx=0x7f004c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:35.902 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:35.905+0000 7f0057400640 1 -- 172.21.15.106:0/2433776931 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0050054060 con 0x7f0060108d90 2024-04-28T12:51:36.111 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.114+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f006002dd80 con 0x7f0060108d90 2024-04-28T12:51:36.112 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.115+0000 7f0057400640 1 -- 172.21.15.106:0/2433776931 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f005009d080 con 0x7f0060108d90 2024-04-28T12:51:36.112 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-28T12:51:36.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f00440a3c40 msgr2=0x7f00440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f00440a3c40 0x7f00440a6100 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f004c031500 tx=0x7f004c005a20 comp rx=0 tx=0).stop 2024-04-28T12:51:36.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0060108d90 msgr2=0x7f006019efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.114 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0060108d90 0x7f006019efc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f005000e9c0 tx=0x7f005000ee80 comp rx=0 tx=0).stop 2024-04-28T12:51:36.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 shutdown_connections 2024-04-28T12:51:36.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f00440a3c40 0x7f00440a6100 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f006010a940 0x7f00601a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0060109f80 0x7f006019f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 --2- 172.21.15.106:0/2433776931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0060108d90 0x7f006019efc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.117+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 >> 172.21.15.106:0/2433776931 conn(0x7f0060077d30 msgr2=0x7f006010d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:36.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.118+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 shutdown_connections 2024-04-28T12:51:36.115 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.118+0000 7f005fe00640 1 -- 172.21.15.106:0/2433776931 wait complete. 2024-04-28T12:51:36.134 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T12:51:36.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.344+0000 7f31a1200640 1 -- 172.21.15.106:0/3977948457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 msgr2=0x7f319c10cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.344+0000 7f31a1200640 1 --2- 172.21.15.106:0/3977948457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 0x7f319c10cfd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3194009be0 tx=0x7f319402a6c0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.344+0000 7f31a1200640 1 -- 172.21.15.106:0/3977948457 shutdown_connections 2024-04-28T12:51:36.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.344+0000 7f31a1200640 1 --2- 172.21.15.106:0/3977948457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f319c105d80 0x7f319c106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.344+0000 7f31a1200640 1 --2- 172.21.15.106:0/3977948457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 0x7f319c10cfd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.344+0000 7f31a1200640 1 --2- 172.21.15.106:0/3977948457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f319c104b90 0x7f319c104fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.344+0000 7f31a1200640 1 -- 172.21.15.106:0/3977948457 >> 172.21.15.106:0/3977948457 conn(0x7f319c1000e0 msgr2=0x7f319c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:36.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.344+0000 7f31a1200640 1 -- 172.21.15.106:0/3977948457 shutdown_connections 2024-04-28T12:51:36.342 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.344+0000 7f31a1200640 1 -- 172.21.15.106:0/3977948457 wait complete. 2024-04-28T12:51:36.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.346+0000 7f31a1200640 1 Processor -- start 2024-04-28T12:51:36.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.346+0000 7f31a1200640 1 -- start start 2024-04-28T12:51:36.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.346+0000 7f31a1200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f319c104b90 0x7f319c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:36.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.346+0000 7f31a1200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f319c105d80 0x7f319c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:36.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f31a1200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 0x7f319c1a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:36.344 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f31a1200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1106c0 con 0x7f319c104b90 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f31a1200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f319c110540 con 0x7f319c106740 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f31a1200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f319c110840 con 0x7f319c105d80 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f319a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 0x7f319c1a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f3199e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f319c104b90 0x7f319c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f3199400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f319c105d80 0x7f319c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f319a800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 0x7f319c1a2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44762/0 (socket says 172.21.15.106:44762) 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f319a800640 1 -- 172.21.15.106:0/589528432 learned_addr learned my addr 172.21.15.106:0/589528432 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f319a800640 1 -- 172.21.15.106:0/589528432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f319c105d80 msgr2=0x7f319c19b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f319a800640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f319c105d80 0x7f319c19b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f319a800640 1 -- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f319c104b90 msgr2=0x7f319c19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f319a800640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f319c104b90 0x7f319c19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f319a800640 1 -- 172.21.15.106:0/589528432 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c1a29c0 con 0x7f319c106740 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.347+0000 7f3199e00640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f319c104b90 0x7f319c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:36.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.348+0000 7f319a800640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 0x7f319c1a2320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3194004af0 tx=0x7f3194030d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:36.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.348+0000 7f3192a00640 1 -- 172.21.15.106:0/589528432 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194041020 con 0x7f319c106740 2024-04-28T12:51:36.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.348+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c1a2c50 con 0x7f319c106740 2024-04-28T12:51:36.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.348+0000 7f3192a00640 1 -- 172.21.15.106:0/589528432 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3194030ec0 con 0x7f319c106740 2024-04-28T12:51:36.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.348+0000 7f3192a00640 1 -- 172.21.15.106:0/589528432 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194036040 con 0x7f319c106740 2024-04-28T12:51:36.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.348+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a3190 con 0x7f319c106740 2024-04-28T12:51:36.347 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.349+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f319c10bf00 con 0x7f319c106740 2024-04-28T12:51:36.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.351+0000 7f3192a00640 1 -- 172.21.15.106:0/589528432 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3194044020 con 0x7f319c106740 2024-04-28T12:51:36.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.352+0000 7f3192a00640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31800a3c40 0x7f31800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:36.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.353+0000 7f3192a00640 1 -- 172.21.15.106:0/589528432 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f31940f1670 con 0x7f319c106740 2024-04-28T12:51:36.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.353+0000 7f3199e00640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31800a3c40 0x7f31800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:36.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.353+0000 7f3199e00640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31800a3c40 0x7f31800a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f319c19c280 tx=0x7f318c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:36.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.357+0000 7f3192a00640 1 -- 172.21.15.106:0/589528432 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f319404b020 con 0x7f319c106740 2024-04-28T12:51:36.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:36 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2017892643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:36.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:36 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1581397541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:36.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:36 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2433776931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:36.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:36 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2017892643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:36.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:36 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1581397541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:36.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:36 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2433776931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:36.554 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.557+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f319c02dc60 con 0x7f319c106740 2024-04-28T12:51:36.555 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.558+0000 7f3192a00640 1 -- 172.21.15.106:0/589528432 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f31940bdae0 con 0x7f319c106740 2024-04-28T12:51:36.555 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T12:51:36.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31800a3c40 msgr2=0x7f31800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31800a3c40 0x7f31800a6100 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f319c19c280 tx=0x7f318c005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:36.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 msgr2=0x7f319c1a2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 0x7f319c1a2320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3194004af0 tx=0x7f3194030d50 comp rx=0 tx=0).stop 2024-04-28T12:51:36.558 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 shutdown_connections 2024-04-28T12:51:36.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f31800a3c40 0x7f31800a6100 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106740 0x7f319c1a2320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f319c105d80 0x7f319c19b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 --2- 172.21.15.106:0/589528432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f319c104b90 0x7f319c19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 >> 172.21.15.106:0/589528432 conn(0x7f319c1000e0 msgr2=0x7f319c1090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:36.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 shutdown_connections 2024-04-28T12:51:36.559 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.561+0000 7f31a1200640 1 -- 172.21.15.106:0/589528432 wait complete. 2024-04-28T12:51:36.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:36 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2017892643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T12:51:36.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:36 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1581397541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T12:51:36.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:36 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2433776931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T12:51:36.577 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T12:51:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.789+0000 7f3a61e00640 1 -- 172.21.15.106:0/1946004062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c105b50 msgr2=0x7f3a5c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.789+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1946004062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c105b50 0x7f3a5c105fd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009860 tx=0x7f3a4802a340 comp rx=0 tx=0).stop 2024-04-28T12:51:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.790+0000 7f3a61e00640 1 -- 172.21.15.106:0/1946004062 shutdown_connections 2024-04-28T12:51:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.790+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1946004062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c105b50 0x7f3a5c105fd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.790+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1946004062 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a5c106510 0x7f3a5c10cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.790+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1946004062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a5c104960 0x7f3a5c104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.790+0000 7f3a61e00640 1 -- 172.21.15.106:0/1946004062 >> 172.21.15.106:0/1946004062 conn(0x7f3a5c0ffeb0 msgr2=0x7f3a5c102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.790+0000 7f3a61e00640 1 -- 172.21.15.106:0/1946004062 shutdown_connections 2024-04-28T12:51:36.787 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.790+0000 7f3a61e00640 1 -- 172.21.15.106:0/1946004062 wait complete. 2024-04-28T12:51:36.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.791+0000 7f3a61e00640 1 Processor -- start 2024-04-28T12:51:36.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.791+0000 7f3a61e00640 1 -- start start 2024-04-28T12:51:36.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.791+0000 7f3a61e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a5c104960 0x7f3a5c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:36.789 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.791+0000 7f3a61e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a5c105b50 0x7f3a5c19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.791+0000 7f3a61e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c106510 0x7f3a5c1a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.791+0000 7f3a61e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c110470 con 0x7f3a5c106510 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.791+0000 7f3a61e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c1102f0 con 0x7f3a5c104960 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.791+0000 7f3a61e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c1105f0 con 0x7f3a5c105b50 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.791+0000 7f3a61400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c106510 0x7f3a5c1a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a5be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a5c105b50 0x7f3a5c19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a61400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c106510 0x7f3a5c1a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45218/0 (socket says 172.21.15.106:45218) 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a61400640 1 -- 172.21.15.106:0/1989157383 learned_addr learned my addr 172.21.15.106:0/1989157383 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a60a00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a5c104960 0x7f3a5c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a5be00640 1 -- 172.21.15.106:0/1989157383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a5c104960 msgr2=0x7f3a5c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a5be00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a5c104960 0x7f3a5c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a5be00640 1 -- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c106510 msgr2=0x7f3a5c1a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:36.790 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a5be00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c106510 0x7f3a5c1a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:36.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a5be00640 1 -- 172.21.15.106:0/1989157383 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c1a27d0 con 0x7f3a5c105b50 2024-04-28T12:51:36.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a61400640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c106510 0x7f3a5c1a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:36.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.792+0000 7f3a5be00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a5c105b50 0x7f3a5c19b0f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3a4802a860 tx=0x7f3a48032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:36.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.793+0000 7f3a59600640 1 -- 172.21.15.106:0/1989157383 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48040020 con 0x7f3a5c105b50 2024-04-28T12:51:36.791 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.793+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c1a2a60 con 0x7f3a5c105b50 2024-04-28T12:51:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.793+0000 7f3a59600640 1 -- 172.21.15.106:0/1989157383 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a48030d50 con 0x7f3a5c105b50 2024-04-28T12:51:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.793+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c1a2fa0 con 0x7f3a5c105b50 2024-04-28T12:51:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.793+0000 7f3a59600640 1 -- 172.21.15.106:0/1989157383 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48037040 con 0x7f3a5c105b50 2024-04-28T12:51:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.794+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c10bcf0 con 0x7f3a5c105b50 2024-04-28T12:51:36.792 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.796+0000 7f3a59600640 1 -- 172.21.15.106:0/1989157383 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3a48043020 con 0x7f3a5c105b50 2024-04-28T12:51:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.798+0000 7f3a59600640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3a440a3cc0 0x7f3a440a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.798+0000 7f3a59600640 1 -- 172.21.15.106:0/1989157383 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f3a480efef0 con 0x7f3a5c105b50 2024-04-28T12:51:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.798+0000 7f3a60a00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3a440a3cc0 0x7f3a440a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:36.795 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.799+0000 7f3a60a00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3a440a3cc0 0x7f3a440a6180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3a500095f0 tx=0x7f3a50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:36.798 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:36.803+0000 7f3a59600640 1 -- 172.21.15.106:0/1989157383 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3a4804a030 con 0x7f3a5c105b50 2024-04-28T12:51:37.045 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.049+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a5c02dd20 con 0x7f3a5c105b50 2024-04-28T12:51:37.046 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.050+0000 7f3a59600640 1 -- 172.21.15.106:0/1989157383 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11953 (secure 0 0 0) 0x7f3a480bc470 con 0x7f3a5c105b50 2024-04-28T12:51:37.046 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:37.046 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:49:30.023676+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24548,"mds_3":24538,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24548,24520,24578,24538,24566]},"id":1}]} 2024-04-28T12:51:37.047 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.052+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3a440a3cc0 msgr2=0x7f3a440a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.052+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3a440a3cc0 0x7f3a440a6180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3a500095f0 tx=0x7f3a50005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a5c105b50 msgr2=0x7f3a5c19b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a5c105b50 0x7f3a5c19b0f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3a4802a860 tx=0x7f3a48032d50 comp rx=0 tx=0).stop 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 shutdown_connections 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3a440a3cc0 0x7f3a440a6180 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a5c106510 0x7f3a5c1a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a5c105b50 0x7f3a5c19b0f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 --2- 172.21.15.106:0/1989157383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a5c104960 0x7f3a5c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 >> 172.21.15.106:0/1989157383 conn(0x7f3a5c0ffeb0 msgr2=0x7f3a5c108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 shutdown_connections 2024-04-28T12:51:37.050 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.053+0000 7f3a61e00640 1 -- 172.21.15.106:0/1989157383 wait complete. 2024-04-28T12:51:37.069 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T12:51:37.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.261+0000 7f3248800640 1 -- 172.21.15.106:0/1745831148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240104b70 msgr2=0x7f3240104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.261+0000 7f3248800640 1 --2- 172.21.15.106:0/1745831148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240104b70 0x7f3240104f90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3234009900 tx=0x7f323402a410 comp rx=0 tx=0).stop 2024-04-28T12:51:37.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.262+0000 7f3248800640 1 -- 172.21.15.106:0/1745831148 shutdown_connections 2024-04-28T12:51:37.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.262+0000 7f3248800640 1 --2- 172.21.15.106:0/1745831148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3240106720 0x7f324010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.262+0000 7f3248800640 1 --2- 172.21.15.106:0/1745831148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3240105d60 0x7f32401061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.258 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.262+0000 7f3248800640 1 --2- 172.21.15.106:0/1745831148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240104b70 0x7f3240104f90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.262+0000 7f3248800640 1 -- 172.21.15.106:0/1745831148 >> 172.21.15.106:0/1745831148 conn(0x7f3240100120 msgr2=0x7f3240102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.262+0000 7f3248800640 1 -- 172.21.15.106:0/1745831148 shutdown_connections 2024-04-28T12:51:37.259 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.262+0000 7f3248800640 1 -- 172.21.15.106:0/1745831148 wait complete. 2024-04-28T12:51:37.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.263+0000 7f3248800640 1 Processor -- start 2024-04-28T12:51:37.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.263+0000 7f3248800640 1 -- start start 2024-04-28T12:51:37.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.263+0000 7f3248800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3240104b70 0x7f324019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.263+0000 7f3248800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240105d60 0x7f324019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.263+0000 7f3248800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3240106720 0x7f32401a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.263+0000 7f3248800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f32401106e0 con 0x7f3240104b70 2024-04-28T12:51:37.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.263+0000 7f3248800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3240110560 con 0x7f3240106720 2024-04-28T12:51:37.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.263+0000 7f3248800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3240110860 con 0x7f3240105d60 2024-04-28T12:51:37.261 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3240104b70 0x7f324019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240105d60 0x7f324019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240105d60 0x7f324019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49758/0 (socket says 172.21.15.106:49758) 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245400640 1 -- 172.21.15.106:0/1655723205 learned_addr learned my addr 172.21.15.106:0/1655723205 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3246800640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3240106720 0x7f32401a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245400640 1 -- 172.21.15.106:0/1655723205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3240106720 msgr2=0x7f32401a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245400640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3240106720 0x7f32401a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245400640 1 -- 172.21.15.106:0/1655723205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3240104b70 msgr2=0x7f324019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245400640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3240104b70 0x7f324019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245400640 1 -- 172.21.15.106:0/1655723205 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32401a2a00 con 0x7f3240105d60 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.264+0000 7f3245400640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240105d60 0x7f324019b320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f323000e9f0 tx=0x7f323000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.265+0000 7f323aa00640 1 -- 172.21.15.106:0/1655723205 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323000cd80 con 0x7f3240105d60 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.265+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32401a2cf0 con 0x7f3240105d60 2024-04-28T12:51:37.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.265+0000 7f323aa00640 1 -- 172.21.15.106:0/1655723205 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32300054d0 con 0x7f3240105d60 2024-04-28T12:51:37.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.265+0000 7f323aa00640 1 -- 172.21.15.106:0/1655723205 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230010690 con 0x7f3240105d60 2024-04-28T12:51:37.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.265+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32401a31e0 con 0x7f3240105d60 2024-04-28T12:51:37.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.266+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324010beb0 con 0x7f3240105d60 2024-04-28T12:51:37.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.270+0000 7f323aa00640 1 -- 172.21.15.106:0/1655723205 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3230004e90 con 0x7f3240105d60 2024-04-28T12:51:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.271+0000 7f323aa00640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f32280a3c40 0x7f32280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.271+0000 7f323aa00640 1 -- 172.21.15.106:0/1655723205 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f32300d0d40 con 0x7f3240105d60 2024-04-28T12:51:37.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.271+0000 7f3245e00640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f32280a3c40 0x7f32280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.272+0000 7f3245e00640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f32280a3c40 0x7f32280a6100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3234005ab0 tx=0x7f3234005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.271 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.276+0000 7f323aa00640 1 -- 172.21.15.106:0/1655723205 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f323009d1f0 con 0x7f3240105d60 2024-04-28T12:51:37.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:37 smithi106 ceph-mon[28081]: pgmap v403: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:51:37.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:37 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/589528432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:37.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:37 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1989157383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:37.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.479+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f324002dd20 con 0x7f3240105d60 2024-04-28T12:51:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.481+0000 7f323aa00640 1 -- 172.21.15.106:0/1655723205 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24928 (secure 0 0 0) 0x7f323009caf0 con 0x7f3240105d60 2024-04-28T12:51:37.478 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T12:51:37.478 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":93,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:50:58.139109+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T12:51:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f32280a3c40 msgr2=0x7f32280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f32280a3c40 0x7f32280a6100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3234005ab0 tx=0x7f3234005a20 comp rx=0 tx=0).stop 2024-04-28T12:51:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240105d60 msgr2=0x7f324019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240105d60 0x7f324019b320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f323000e9f0 tx=0x7f323000edf0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 shutdown_connections 2024-04-28T12:51:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f32280a3c40 0x7f32280a6100 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3240106720 0x7f32401a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3240105d60 0x7f324019b320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 --2- 172.21.15.106:0/1655723205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3240104b70 0x7f324019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 >> 172.21.15.106:0/1655723205 conn(0x7f3240100120 msgr2=0x7f3240109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 shutdown_connections 2024-04-28T12:51:37.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.484+0000 7f3248800640 1 -- 172.21.15.106:0/1655723205 wait complete. 2024-04-28T12:51:37.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:37 smithi171 ceph-mon[63127]: pgmap v403: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:51:37.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:37 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/589528432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:37.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:37 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1989157383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:37.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-28T12:48:54.598061+0000', 'modified': '2024-04-28T12:49:30.023676+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24578, 'mds_1': 24566, 'mds_2': 24548, 'mds_3': 24538, 'mds_4': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24469': {'gid': 24469, 'name': 'j', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6835/3035871722', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 3035871722}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 3035871722}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'd', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/4131098763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 4131098763}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 4131098763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6835/1242652051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 1242652051}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 1242652051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24508': {'gid': 24508, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6835/4221893526', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 4221893526}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 4221893526}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6837/1065884016', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 1065884016}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 1065884016}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24520': {'gid': 24520, 'name': 'c', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.171:6837/813345311', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 813345311}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 813345311}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.171:6839/2406303763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 2406303763}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 2406303763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'g', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.106:6839/3863798001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 3863798001}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 3863798001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'a', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.106:6841/92504617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6840', 'nonce': 92504617}, {'type': 'v1', 'addr': '172.21.15.106:6841', 'nonce': 92504617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24578': {'gid': 24578, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.129:6839/4208792665', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 4208792665}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 4208792665}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24578, 'qdb_cluster': [24548, 24520, 24578, 24538, 24566]} 2024-04-28T12:51:37.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-28T12:51:37.499 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-28T12:51:37.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:37 smithi129 ceph-mon[62418]: pgmap v403: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:51:37.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:37 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/589528432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T12:51:37.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:37 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1989157383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T12:51:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.682+0000 7f69ae000640 1 -- 172.21.15.106:0/2196599648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a81049b0 msgr2=0x7f69a8104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.682+0000 7f69ae000640 1 --2- 172.21.15.106:0/2196599648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a81049b0 0x7f69a8104dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f699c009900 tx=0x7f699c02a3f0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.682+0000 7f69ae000640 1 -- 172.21.15.106:0/2196599648 shutdown_connections 2024-04-28T12:51:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.682+0000 7f69ae000640 1 --2- 172.21.15.106:0/2196599648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69a8105ba0 0x7f69a8106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.682+0000 7f69ae000640 1 --2- 172.21.15.106:0/2196599648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69a8106560 0x7f69a810cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.682+0000 7f69ae000640 1 --2- 172.21.15.106:0/2196599648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a81049b0 0x7f69a8104dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.682+0000 7f69ae000640 1 -- 172.21.15.106:0/2196599648 >> 172.21.15.106:0/2196599648 conn(0x7f69a80fff40 msgr2=0x7f69a8102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.682+0000 7f69ae000640 1 -- 172.21.15.106:0/2196599648 shutdown_connections 2024-04-28T12:51:37.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.682+0000 7f69ae000640 1 -- 172.21.15.106:0/2196599648 wait complete. 2024-04-28T12:51:37.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.683+0000 7f69ae000640 1 Processor -- start 2024-04-28T12:51:37.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.683+0000 7f69ae000640 1 -- start start 2024-04-28T12:51:37.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.683+0000 7f69ae000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69a81049b0 0x7f69a819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.683+0000 7f69ae000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a8105ba0 0x7f69a819b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.681 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69ae000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69a8106560 0x7f69a81a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69ae000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8110500 con 0x7f69a81049b0 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69ae000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8110380 con 0x7f69a8105ba0 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69ae000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8110680 con 0x7f69a8106560 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69a7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a8105ba0 0x7f69a819b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69acc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69a81049b0 0x7f69a819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69a7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a8105ba0 0x7f69a819b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44822/0 (socket says 172.21.15.106:44822) 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69acc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69a81049b0 0x7f69a819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45242/0 (socket says 172.21.15.106:45242) 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69a7e00640 1 -- 172.21.15.106:0/915808880 learned_addr learned my addr 172.21.15.106:0/915808880 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69ad600640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69a8106560 0x7f69a81a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69a7e00640 1 -- 172.21.15.106:0/915808880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69a8106560 msgr2=0x7f69a81a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69a7e00640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69a8106560 0x7f69a81a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69a7e00640 1 -- 172.21.15.106:0/915808880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69a81049b0 msgr2=0x7f69a819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69a7e00640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69a81049b0 0x7f69a819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.684+0000 7f69a7e00640 1 -- 172.21.15.106:0/915808880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a81a27b0 con 0x7f69a8105ba0 2024-04-28T12:51:37.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.685+0000 7f69a7e00640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a8105ba0 0x7f69a819b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f699400e9f0 tx=0x7f699400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.685+0000 7f69a5600640 1 -- 172.21.15.106:0/915808880 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699400cd80 con 0x7f69a8105ba0 2024-04-28T12:51:37.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.685+0000 7f69ae000640 1 -- 172.21.15.106:0/915808880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a81a2aa0 con 0x7f69a8105ba0 2024-04-28T12:51:37.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.685+0000 7f69a5600640 1 -- 172.21.15.106:0/915808880 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69940108e0 con 0x7f69a8105ba0 2024-04-28T12:51:37.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.685+0000 7f69a5600640 1 -- 172.21.15.106:0/915808880 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994018980 con 0x7f69a8105ba0 2024-04-28T12:51:37.683 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.685+0000 7f69ae000640 1 -- 172.21.15.106:0/915808880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a81a2ff0 con 0x7f69a8105ba0 2024-04-28T12:51:37.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.688+0000 7f69b25eb740 1 Processor -- start 2024-04-28T12:51:37.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.688+0000 7f69b25eb740 1 -- start start 2024-04-28T12:51:37.684 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.688+0000 7f69a5600640 1 -- 172.21.15.106:0/915808880 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f6994018b20 con 0x7f69a8105ba0 2024-04-28T12:51:37.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.688+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.688+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.688+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.688+0000 7f69b25eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f49742d600 con 0x55f4974cee20 2024-04-28T12:51:37.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.688+0000 7f69b25eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f4973ca100 con 0x55f4974c92c0 2024-04-28T12:51:37.685 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69b25eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f4974295c0 con 0x55f4974c9cb0 2024-04-28T12:51:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69a7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69ad600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69ad600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44832/0 (socket says 172.21.15.106:44832) 2024-04-28T12:51:37.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69ad600640 1 -- 172.21.15.106:0/2199721188 learned_addr learned my addr 172.21.15.106:0/2199721188 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69a7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45246/0 (socket says 172.21.15.106:45246) 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69acc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69ad600640 1 -- 172.21.15.106:0/2199721188 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 msgr2=0x55f4974ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69ad600640 1 --2- 172.21.15.106:0/2199721188 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69ad600640 1 -- 172.21.15.106:0/2199721188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 msgr2=0x55f4974d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69ad600640 1 --2- 172.21.15.106:0/2199721188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.689+0000 7f69ad600640 1 -- 172.21.15.106:0/2199721188 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49713f2d0 con 0x55f4974c92c0 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.690+0000 7f69ad600640 1 --2- 172.21.15.106:0/2199721188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974c96e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f69a0010370 tx=0x7f69a0010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13e22e7c7d6362f server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.690+0000 7f6999c00640 1 -- 172.21.15.106:0/2199721188 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0016070 con 0x55f4974c92c0 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.690+0000 7f6999c00640 1 -- 172.21.15.106:0/2199721188 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a0004d20 con 0x55f4974c92c0 2024-04-28T12:51:37.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.690+0000 7f6999c00640 1 -- 172.21.15.106:0/2199721188 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a0005420 con 0x55f4974c92c0 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69b25eb740 1 -- 172.21.15.106:0/2199721188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 msgr2=0x55f4974c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69b25eb740 1 --2- 172.21.15.106:0/2199721188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974c96e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f69a0010370 tx=0x7f69a0010830 comp rx=0 tx=0).stop 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69b25eb740 1 -- 172.21.15.106:0/2199721188 shutdown_connections 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69b25eb740 1 --2- 172.21.15.106:0/2199721188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69b25eb740 1 --2- 172.21.15.106:0/2199721188 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69b25eb740 1 --2- 172.21.15.106:0/2199721188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974c96e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69b25eb740 1 -- 172.21.15.106:0/2199721188 >> 172.21.15.106:0/2199721188 conn(0x55f49745dd10 msgr2=0x55f4974c4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69b25eb740 1 -- 172.21.15.106:0/2199721188 shutdown_connections 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69a5600640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f69900a3b50 0x7f69900a6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.691+0000 7f69b25eb740 1 -- 172.21.15.106:0/2199721188 wait complete. 2024-04-28T12:51:37.692 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69a5600640 1 -- 172.21.15.106:0/915808880 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f69940d0c90 con 0x7f69a8105ba0 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69acc00640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f69900a3b50 0x7f69900a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69b25eb740 1 -- start start 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69b25eb740 1 Processor -- start 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c92c0 0x55f4974d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c9cb0 0x55f4974d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 0x55f4974dac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69b25eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f4974d1dd0 con 0x55f4974c9cb0 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69b25eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f4973b4f70 con 0x55f4974cee20 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69b25eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f4974d1f50 con 0x55f4974c92c0 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.692+0000 7f69acc00640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f69900a3b50 0x7f69900a6010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f699c005ab0 tx=0x7f699c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69b25eb740 10 client.0 _ll_register_callbacks cb 0x55f497423480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69a7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c9cb0 0x55f4974d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69acc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c92c0 0x55f4974d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69a7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c9cb0 0x55f4974d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45262/0 (socket says 172.21.15.106:45262) 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69a7e00640 1 -- 172.21.15.106:0/1473866819 learned_addr learned my addr 172.21.15.106:0/1473866819 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69ad600640 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 0x55f4974dac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69a7e00640 1 -- 172.21.15.106:0/1473866819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c92c0 msgr2=0x55f4974d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69a7e00640 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c92c0 0x55f4974d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69a7e00640 1 -- 172.21.15.106:0/1473866819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 msgr2=0x55f4974dac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69a7e00640 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 0x55f4974dac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69a7e00640 1 -- 172.21.15.106:0/1473866819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4972096b0 con 0x55f4974c9cb0 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69acc00640 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c92c0 0x55f4974d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f69a7e00640 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c9cb0 0x55f4974d3bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f699409ca40 tx=0x7f69940d4330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae148b2ed55fd56 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.693+0000 7f698e000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.694+0000 7f698e000640 1 -- 172.21.15.106:0/1473866819 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699409cab0 con 0x55f4974c9cb0 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.694+0000 7f698e000640 1 -- 172.21.15.106:0/1473866819 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f69940bbbc0 con 0x55f4974c9cb0 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.694+0000 7f69b25eb740 1 -- 172.21.15.106:0/1473866819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69a0004d20 con 0x55f4974c9cb0 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.694+0000 7f698e000640 1 -- 172.21.15.106:0/1473866819 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f69940a0db0 con 0x55f4974c9cb0 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.694+0000 7f69b25eb740 10 client.15261 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.694+0000 7f69b25eb740 1 -- 172.21.15.106:0/1473866819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4974ddc80 con 0x55f4974c9cb0 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.694+0000 7f698e000640 1 -- 172.21.15.106:0/1473866819 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a0004d20 con 0x55f4974c9cb0 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.694+0000 7f698e000640 1 -- 172.21.15.106:0/1473866819 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f699409bf00 con 0x55f4974c9cb0 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.695+0000 7f69b25eb740 10 client.15261 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.695+0000 7f69b25eb740 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.695+0000 7f69b25eb740 20 client.15261 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.695+0000 7f69b25eb740 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55f4974e02d0 0x55f4974e1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.695+0000 7f69b25eb740 4 client.15261 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.695+0000 7f69b25eb740 1 -- 172.21.15.106:0/1473866819 --> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f49743b210 con 0x55f4974e02d0 2024-04-28T12:51:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.695+0000 7f69acc00640 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55f4974e02d0 0x55f4974e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.696+0000 7f69acc00640 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55f4974e02d0 0x55f4974e1740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da2307d3fb252f86 server_cookie=a8526daa55f46afc in_seq=0 out_seq=0 2024-04-28T12:51:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.696+0000 7f698e000640 10 client.15261 ms_handle_connect on v2:172.21.15.171:6836/813345311 2024-04-28T12:51:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.696+0000 7f698e000640 1 -- 172.21.15.106:0/1473866819 <== mds.4 v2:172.21.15.171:6836/813345311 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f49743b210 con 0x55f4974e02d0 2024-04-28T12:51:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.696+0000 7f698e000640 10 client.15261 handle_command_reply: tid=0 2024-04-28T12:51:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.696+0000 7f69b25eb740 1 client.15261 shutdown 2024-04-28T12:51:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 -- 172.21.15.106:0/1473866819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c9cb0 msgr2=0x55f4974d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c9cb0 0x55f4974d3bf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f699409ca40 tx=0x7f69940d4330 comp rx=0 tx=0).stop 2024-04-28T12:51:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 -- 172.21.15.106:0/1473866819 shutdown_connections 2024-04-28T12:51:37.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55f4974e02d0 0x55f4974e1740 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c9cb0 0x55f4974d3bf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 0x55f4974dac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 --2- 172.21.15.106:0/1473866819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c92c0 0x55f4974d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 -- 172.21.15.106:0/1473866819 >> 172.21.15.106:0/1473866819 conn(0x55f49745dd10 msgr2=0x55f4974c3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f698e000640 0 client.15261 ms_handle_reset on v2:172.21.15.171:6836/813345311 2024-04-28T12:51:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 -- 172.21.15.106:0/1473866819 shutdown_connections 2024-04-28T12:51:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 1 -- 172.21.15.106:0/1473866819 wait complete. 2024-04-28T12:51:37.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.697+0000 7f69b25eb740 20 client.15261 trim_cache size 0 max 16384 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.717+0000 7f69b25eb740 1 Processor -- start 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.717+0000 7f69b25eb740 1 -- start start 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.717+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.717+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974dabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.717+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.717+0000 7f69b25eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f4974dfa80 con 0x55f4974cee20 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.717+0000 7f69b25eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f4974e0150 con 0x55f4974c92c0 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.717+0000 7f69b25eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f4974e0820 con 0x55f4974c9cb0 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69ad600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69a7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974dabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69acc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69a7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974dabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49796/0 (socket says 172.21.15.106:49796) 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69ad600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45264/0 (socket says 172.21.15.106:45264) 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69a7e00640 1 -- 172.21.15.106:0/28669590 learned_addr learned my addr 172.21.15.106:0/28669590 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69a7e00640 1 -- 172.21.15.106:0/28669590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 msgr2=0x55f4974ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69a7e00640 1 --2- 172.21.15.106:0/28669590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69a7e00640 1 -- 172.21.15.106:0/28669590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 msgr2=0x55f4974d3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69a7e00640 1 --2- 172.21.15.106:0/28669590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69a7e00640 1 -- 172.21.15.106:0/28669590 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4974c52a0 con 0x55f4974c9cb0 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.718+0000 7f69acc00640 1 --2- 172.21.15.106:0/28669590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.719+0000 7f69a7e00640 1 --2- 172.21.15.106:0/28669590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974dabf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f69940d2040 tx=0x7f6994000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18a9a4fbf076ffcb server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.719+0000 7f6999200640 1 -- 172.21.15.106:0/28669590 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699409c5f0 con 0x55f4974c9cb0 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.719+0000 7f6999200640 1 -- 172.21.15.106:0/28669590 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69940b5070 con 0x55f4974c9cb0 2024-04-28T12:51:37.716 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.719+0000 7f6999200640 1 -- 172.21.15.106:0/28669590 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69940ce480 con 0x55f4974c9cb0 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 -- 172.21.15.106:0/28669590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 msgr2=0x55f4974dabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 --2- 172.21.15.106:0/28669590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974dabf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f69940d2040 tx=0x7f6994000bc0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 -- 172.21.15.106:0/28669590 shutdown_connections 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 --2- 172.21.15.106:0/28669590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974cee20 0x55f4974d3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 --2- 172.21.15.106:0/28669590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974dabf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 --2- 172.21.15.106:0/28669590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974c92c0 0x55f4974ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 -- 172.21.15.106:0/28669590 >> 172.21.15.106:0/28669590 conn(0x55f49745dd10 msgr2=0x55f4974cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 -- 172.21.15.106:0/28669590 shutdown_connections 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 -- 172.21.15.106:0/28669590 wait complete. 2024-04-28T12:51:37.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 -- start start 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.720+0000 7f69b25eb740 1 Processor -- start 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c92c0 0x55f4974d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974d92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69b25eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 0x55f4974d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69b25eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f4974e15c0 con 0x55f4974c92c0 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69b25eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f4974e0ef0 con 0x55f4974cee20 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69b25eb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f4974e1c90 con 0x55f4974c9cb0 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69b25eb740 10 client.0 _ll_register_callbacks cb 0x7f69940d4430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69acc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c92c0 0x55f4974d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69acc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c92c0 0x55f4974d8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45280/0 (socket says 172.21.15.106:45280) 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69acc00640 1 -- 172.21.15.106:0/596869159 learned_addr learned my addr 172.21.15.106:0/596869159 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69ad600640 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 0x55f4974d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69a7e00640 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974d92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69acc00640 1 -- 172.21.15.106:0/596869159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 msgr2=0x55f4974d92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69acc00640 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974d92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69acc00640 1 -- 172.21.15.106:0/596869159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 msgr2=0x55f4974d9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69acc00640 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 0x55f4974d9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.721+0000 7f69acc00640 1 -- 172.21.15.106:0/596869159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4974d9f80 con 0x55f4974c92c0 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.722+0000 7f69acc00640 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c92c0 0x55f4974d8d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f699c0096c0 tx=0x7f699c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c4dd166a0e3729 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.722+0000 7f698e000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.722+0000 7f698e000640 1 -- 172.21.15.106:0/596869159 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c057070 con 0x55f4974c92c0 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.722+0000 7f698e000640 1 -- 172.21.15.106:0/596869159 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f699c0535b0 con 0x55f4974c92c0 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.722+0000 7f69b25eb740 1 -- 172.21.15.106:0/596869159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69940b5070 con 0x55f4974c92c0 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.722+0000 7f698e000640 1 -- 172.21.15.106:0/596869159 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f699c063050 con 0x55f4974c92c0 2024-04-28T12:51:37.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.722+0000 7f69b25eb740 10 client.15267 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.722+0000 7f69b25eb740 1 -- 172.21.15.106:0/596869159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f4975463b0 con 0x55f4974c92c0 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.723+0000 7f698e000640 1 -- 172.21.15.106:0/596869159 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69940b5070 con 0x55f4974c92c0 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.723+0000 7f698e000640 1 -- 172.21.15.106:0/596869159 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f699c044040 con 0x55f4974c92c0 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.723+0000 7f69b25eb740 10 client.15267 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.723+0000 7f69b25eb740 10 client.15267 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.723+0000 7f69b25eb740 20 client.15267 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.723+0000 7f69b25eb740 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55f49754af60 0x55f497549c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.723+0000 7f69b25eb740 4 client.15267 mds_command: new command op to 24520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.723+0000 7f69b25eb740 1 -- 172.21.15.106:0/596869159 --> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] -- command(tid 0: {"prefix": "status"}) -- 0x55f497435080 con 0x55f49754af60 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.723+0000 7f69a7e00640 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55f49754af60 0x55f497549c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69a7e00640 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55f49754af60 0x55f497549c70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95c89f98f5abd269 server_cookie=36213be527423a04 in_seq=0 out_seq=0 2024-04-28T12:51:37.725 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f698e000640 10 client.15267 ms_handle_connect on v2:172.21.15.171:6836/813345311 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f698e000640 1 -- 172.21.15.106:0/596869159 <== mds.4 v2:172.21.15.171:6836/813345311 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55f497435080 con 0x55f49754af60 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f698e000640 10 client.15267 handle_command_reply: tid=0 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69b25eb740 1 client.15267 shutdown 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69b25eb740 1 -- 172.21.15.106:0/596869159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c92c0 msgr2=0x55f4974d8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69b25eb740 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c92c0 0x55f4974d8d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f699c0096c0 tx=0x7f699c009260 comp rx=0 tx=0).stop 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69b25eb740 1 -- 172.21.15.106:0/596869159 shutdown_connections 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69b25eb740 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.171:6836/813345311,v1:172.21.15.171:6837/813345311] conn(0x55f49754af60 0x55f497549c70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69b25eb740 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4974cee20 0x55f4974d9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69b25eb740 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f4974c9cb0 0x55f4974d92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69b25eb740 1 --2- 172.21.15.106:0/596869159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f4974c92c0 0x55f4974d8d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f69b25eb740 1 -- 172.21.15.106:0/596869159 >> 172.21.15.106:0/596869159 conn(0x55f49745dd10 msgr2=0x55f4974d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.724+0000 7f698e000640 0 client.15267 ms_handle_reset on v2:172.21.15.171:6836/813345311 2024-04-28T12:51:37.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69b25eb740 1 -- 172.21.15.106:0/596869159 shutdown_connections 2024-04-28T12:51:37.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69b25eb740 1 -- 172.21.15.106:0/596869159 wait complete. 2024-04-28T12:51:37.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69b25eb740 20 client.15267 trim_cache size 0 max 16384 2024-04-28T12:51:37.727 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:37.727 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:37.727 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 4, 2024-04-28T12:51:37.727 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24520, 2024-04-28T12:51:37.727 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:37.727 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:37.728 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:37.728 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 128.708140547, 2024-04-28T12:51:37.728 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:37.728 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:37.728 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:37.728 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 185.741060406 2024-04-28T12:51:37.728 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 -- 172.21.15.106:0/915808880 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f69900a3b50 msgr2=0x7f69900a6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f69900a3b50 0x7f69900a6010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f699c005ab0 tx=0x7f699c005a20 comp rx=0 tx=0).stop 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 -- 172.21.15.106:0/915808880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a8105ba0 msgr2=0x7f69a819b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a8105ba0 0x7f69a819b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f699400e9f0 tx=0x7f699400eeb0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 -- 172.21.15.106:0/915808880 shutdown_connections 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f69900a3b50 0x7f69900a6010 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69a8106560 0x7f69a81a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69a8105ba0 0x7f69a819b0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 --2- 172.21.15.106:0/915808880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69a81049b0 0x7f69a819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.725+0000 7f69ae000640 1 -- 172.21.15.106:0/915808880 >> 172.21.15.106:0/915808880 conn(0x7f69a80fff40 msgr2=0x7f69a8108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.726+0000 7f69ae000640 1 -- 172.21.15.106:0/915808880 shutdown_connections 2024-04-28T12:51:37.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.726+0000 7f69ae000640 1 -- 172.21.15.106:0/915808880 wait complete. 2024-04-28T12:51:37.738 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-28T12:51:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.918+0000 7fe7aa800640 1 -- 172.21.15.106:0/638504254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4103490 msgr2=0x7fe7a4105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.918+0000 7fe7aa800640 1 --2- 172.21.15.106:0/638504254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4103490 0x7fe7a4105880 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe794009900 tx=0x7fe79402a410 comp rx=0 tx=0).stop 2024-04-28T12:51:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.918+0000 7fe7aa800640 1 -- 172.21.15.106:0/638504254 shutdown_connections 2024-04-28T12:51:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.918+0000 7fe7aa800640 1 --2- 172.21.15.106:0/638504254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7a41086f0 0x7fe7a410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.918+0000 7fe7aa800640 1 --2- 172.21.15.106:0/638504254 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a4105dc0 0x7fe7a41081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.918+0000 7fe7aa800640 1 --2- 172.21.15.106:0/638504254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4103490 0x7fe7a4105880 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.918+0000 7fe7aa800640 1 -- 172.21.15.106:0/638504254 >> 172.21.15.106:0/638504254 conn(0x7fe7a40fce80 msgr2=0x7fe7a40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.918+0000 7fe7aa800640 1 -- 172.21.15.106:0/638504254 shutdown_connections 2024-04-28T12:51:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.919+0000 7fe7aa800640 1 -- 172.21.15.106:0/638504254 wait complete. 2024-04-28T12:51:37.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.919+0000 7fe7aa800640 1 Processor -- start 2024-04-28T12:51:37.917 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7aa800640 1 -- start start 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7aa800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7a4103490 0x7fe7a419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7aa800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4105dc0 0x7fe7a419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7aa800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a41086f0 0x7fe7a41a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7aa800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a410e540 con 0x7fe7a4105dc0 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7aa800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a410e3c0 con 0x7fe7a4103490 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7aa800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a410e6c0 con 0x7fe7a41086f0 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7a2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4105dc0 0x7fe7a419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7a4103490 0x7fe7a419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7a2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4105dc0 0x7fe7a419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45286/0 (socket says 172.21.15.106:45286) 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7a2a00640 1 -- 172.21.15.106:0/2266365540 learned_addr learned my addr 172.21.15.106:0/2266365540 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7a4103490 0x7fe7a419ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44860/0 (socket says 172.21.15.106:44860) 2024-04-28T12:51:37.918 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.920+0000 7fe7a3e00640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a41086f0 0x7fe7a41a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe7a2a00640 1 -- 172.21.15.106:0/2266365540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a41086f0 msgr2=0x7fe7a41a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a41086f0 0x7fe7a41a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe7a2a00640 1 -- 172.21.15.106:0/2266365540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7a4103490 msgr2=0x7fe7a419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7a4103490 0x7fe7a419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe7a2a00640 1 -- 172.21.15.106:0/2266365540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a41a2a80 con 0x7fe7a4105dc0 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4105dc0 0x7fe7a419b3a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e9c0 tx=0x7fe79000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe79be00640 1 -- 172.21.15.106:0/2266365540 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79000cd80 con 0x7fe7a4105dc0 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe79be00640 1 -- 172.21.15.106:0/2266365540 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7900054d0 con 0x7fe7a4105dc0 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe79be00640 1 -- 172.21.15.106:0/2266365540 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790010430 con 0x7fe7a4105dc0 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe7aa800640 1 -- 172.21.15.106:0/2266365540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a41a2d70 con 0x7fe7a4105dc0 2024-04-28T12:51:37.919 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.921+0000 7fe7aa800640 1 -- 172.21.15.106:0/2266365540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a41a3280 con 0x7fe7a4105dc0 2024-04-28T12:51:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.924+0000 7fe7ad76f740 1 Processor -- start 2024-04-28T12:51:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7ad76f740 1 -- start start 2024-04-28T12:51:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 0x562c62dbb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dc05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dc0c50 0x562c62dc30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c62d1f600 con 0x562c62dc0c50 2024-04-28T12:51:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c62cbc100 con 0x562c62dbbae0 2024-04-28T12:51:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562c62d1b5c0 con 0x562c62dbb0f0 2024-04-28T12:51:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe79be00640 1 -- 172.21.15.106:0/2266365540 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe790004fd0 con 0x7fe7a4105dc0 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dc05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7a3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 0x562c62dbb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dc05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44868/0 (socket says 172.21.15.106:44868) 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.925+0000 7fe7a3400640 1 -- 172.21.15.106:0/3340700350 learned_addr learned my addr 172.21.15.106:0/3340700350 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.926+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/3340700350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dc0c50 0x562c62dc30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.926+0000 7fe7a3400640 1 -- 172.21.15.106:0/3340700350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 msgr2=0x562c62dbb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.926+0000 7fe7a3400640 1 --2- 172.21.15.106:0/3340700350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 0x562c62dbb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.926+0000 7fe7a3400640 1 -- 172.21.15.106:0/3340700350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dc0c50 msgr2=0x562c62dc30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.926+0000 7fe7a3400640 1 --2- 172.21.15.106:0/3340700350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dc0c50 0x562c62dc30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.926+0000 7fe7a3400640 1 -- 172.21.15.106:0/3340700350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c62a312d0 con 0x562c62dbbae0 2024-04-28T12:51:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.926+0000 7fe7a3400640 1 --2- 172.21.15.106:0/3340700350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dc05e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe794009900 tx=0x7fe794005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=626863b5dd346026 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.927+0000 7fe798c00640 1 -- 172.21.15.106:0/3340700350 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794035030 con 0x562c62dbbae0 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.927+0000 7fe798c00640 1 -- 172.21.15.106:0/3340700350 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe794040070 con 0x562c62dbbae0 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.927+0000 7fe798c00640 1 -- 172.21.15.106:0/3340700350 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794003850 con 0x562c62dbbae0 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.927+0000 7fe79be00640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7880a3cc0 0x7fe7880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.927+0000 7fe79be00640 1 -- 172.21.15.106:0/2266365540 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fe7900d0c80 con 0x7fe7a4105dc0 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.927+0000 7fe7a3e00640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7880a3cc0 0x7fe7880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.927+0000 7fe7ad76f740 1 -- 172.21.15.106:0/3340700350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 msgr2=0x562c62dc05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.927+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/3340700350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dc05e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe794009900 tx=0x7fe794005df0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.928+0000 7fe7ad76f740 1 -- 172.21.15.106:0/3340700350 shutdown_connections 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.928+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/3340700350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dc0c50 0x562c62dc30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.928+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/3340700350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dc05e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.928+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/3340700350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 0x562c62dbb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.928+0000 7fe7ad76f740 1 -- 172.21.15.106:0/3340700350 >> 172.21.15.106:0/3340700350 conn(0x562c62d4fd10 msgr2=0x562c62db6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.928+0000 7fe7ad76f740 1 -- 172.21.15.106:0/3340700350 shutdown_connections 2024-04-28T12:51:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.928+0000 7fe7ad76f740 1 -- 172.21.15.106:0/3340700350 wait complete. 2024-04-28T12:51:37.926 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.928+0000 7fe7a3e00640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7880a3cc0 0x7fe7880a6180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x562c62dbc340 tx=0x7fe79c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.928+0000 7fe7ad76f740 1 -- start start 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7ad76f740 1 Processor -- start 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 0x562c62dcdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dc0c50 0x562c62dd5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c62dc3ba0 con 0x562c62dbbae0 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c62ca6f70 con 0x562c62dc0c50 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562c62dc3d20 con 0x562c62dbb0f0 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7ad76f740 10 client.0 _ll_register_callbacks cb 0x562c62d15480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7a3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dc0c50 0x562c62dd5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7a2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.930 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7a3e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dc0c50 0x562c62dd5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44876/0 (socket says 172.21.15.106:44876) 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7a3e00640 1 -- 172.21.15.106:0/636490496 learned_addr learned my addr 172.21.15.106:0/636490496 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.929+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45298/0 (socket says 172.21.15.106:45298) 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe7a3400640 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 0x562c62dcdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe7a2a00640 1 -- 172.21.15.106:0/636490496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 msgr2=0x562c62dcdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 0x562c62dcdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe7a2a00640 1 -- 172.21.15.106:0/636490496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dc0c50 msgr2=0x562c62dd5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dc0c50 0x562c62dd5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe7a2a00640 1 -- 172.21.15.106:0/636490496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c62afb6b0 con 0x562c62dbbae0 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe7a3e00640 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dc0c50 0x562c62dd5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dce320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe7900ce260 tx=0x7fe79009c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb792060ac97cc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe78d600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe78d600640 1 -- 172.21.15.106:0/636490496 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900046c0 con 0x562c62dbbae0 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe78d600640 1 -- 172.21.15.106:0/636490496 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fe7900990a0 con 0x562c62dbbae0 2024-04-28T12:51:37.931 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.930+0000 7fe7ad76f740 1 -- 172.21.15.106:0/636490496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe794040070 con 0x562c62dbbae0 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe78d600640 1 -- 172.21.15.106:0/636490496 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe79009d130 con 0x562c62dbbae0 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe7ad76f740 10 client.15279 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe7ad76f740 1 -- 172.21.15.106:0/636490496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c62dd8400 con 0x562c62dbbae0 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe78d600640 1 -- 172.21.15.106:0/636490496 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe794040070 con 0x562c62dbbae0 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe78d600640 1 -- 172.21.15.106:0/636490496 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe7900a5210 con 0x562c62dbbae0 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe7ad76f740 10 client.15279 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe7ad76f740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe7ad76f740 20 client.15279 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x562c62ddaae0 0x562c62ddbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe7ad76f740 4 client.15279 mds_command: new command op to 24538 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.931+0000 7fe7ad76f740 1 -- 172.21.15.106:0/636490496 --> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c62d2d210 con 0x562c62ddaae0 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.932+0000 7fe7a3400640 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x562c62ddaae0 0x562c62ddbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.932+0000 7fe7a3400640 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x562c62ddaae0 0x562c62ddbf50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=80d00417ae3b84f7 server_cookie=ebd92ac0cdaaa133 in_seq=0 out_seq=0 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.932+0000 7fe78d600640 10 client.15279 ms_handle_connect on v2:172.21.15.171:6838/2406303763 2024-04-28T12:51:37.932 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe78d600640 1 -- 172.21.15.106:0/636490496 <== mds.3 v2:172.21.15.171:6838/2406303763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c62d2d210 con 0x562c62ddaae0 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe78d600640 10 client.15279 handle_command_reply: tid=0 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 client.15279 shutdown 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 -- 172.21.15.106:0/636490496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 msgr2=0x562c62dce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dce320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe7900ce260 tx=0x7fe79009c800 comp rx=0 tx=0).stop 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 -- 172.21.15.106:0/636490496 shutdown_connections 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x562c62ddaae0 0x562c62ddbf50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dce320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dc0c50 0x562c62dd5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/636490496 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dbb0f0 0x562c62dcdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe78d600640 0 client.15279 ms_handle_reset on v2:172.21.15.171:6838/2406303763 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 -- 172.21.15.106:0/636490496 >> 172.21.15.106:0/636490496 conn(0x562c62d4fd10 msgr2=0x562c62db5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.933+0000 7fe7ad76f740 1 -- 172.21.15.106:0/636490496 shutdown_connections 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.934+0000 7fe7ad76f740 1 -- 172.21.15.106:0/636490496 wait complete. 2024-04-28T12:51:37.933 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.934+0000 7fe7ad76f740 20 client.15279 trim_cache size 0 max 16384 2024-04-28T12:51:37.950 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.953+0000 7fe7ad76f740 1 Processor -- start 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7ad76f740 1 -- start start 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbb0f0 0x562c62dcdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dd5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 0x562c62dcdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c62dda290 con 0x562c62dbb0f0 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c62dda960 con 0x562c62dbbae0 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562c62ddb030 con 0x562c62dc0c50 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbb0f0 0x562c62dcdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbb0f0 0x562c62dcdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45302/0 (socket says 172.21.15.106:45302) 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7a3400640 1 -- 172.21.15.106:0/1129855351 learned_addr learned my addr 172.21.15.106:0/1129855351 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/1129855351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dd5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.954+0000 7fe7a3e00640 1 --2- 172.21.15.106:0/1129855351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 0x562c62dcdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.951 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.955+0000 7fe7a3400640 1 -- 172.21.15.106:0/1129855351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 msgr2=0x562c62dcdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.955+0000 7fe7a3400640 1 --2- 172.21.15.106:0/1129855351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 0x562c62dcdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.955+0000 7fe7a3400640 1 -- 172.21.15.106:0/1129855351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 msgr2=0x562c62dd5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.955+0000 7fe7a3400640 1 --2- 172.21.15.106:0/1129855351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dd5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.955+0000 7fe7a3400640 1 -- 172.21.15.106:0/1129855351 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c62dbc370 con 0x562c62dbb0f0 2024-04-28T12:51:37.952 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.955+0000 7fe7a3400640 1 --2- 172.21.15.106:0/1129855351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbb0f0 0x562c62dcdab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe79402a930 tx=0x7fe794000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed899e6615f2a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.955+0000 7fe78fe00640 1 -- 172.21.15.106:0/1129855351 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794035040 con 0x562c62dbb0f0 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.955+0000 7fe78fe00640 1 -- 172.21.15.106:0/1129855351 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe794040040 con 0x562c62dbb0f0 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.955+0000 7fe78fe00640 1 -- 172.21.15.106:0/1129855351 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794055670 con 0x562c62dbb0f0 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 -- 172.21.15.106:0/1129855351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbb0f0 msgr2=0x562c62dcdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/1129855351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbb0f0 0x562c62dcdab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe79402a930 tx=0x7fe794000bc0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 -- 172.21.15.106:0/1129855351 shutdown_connections 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/1129855351 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 0x562c62dcdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/1129855351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbbae0 0x562c62dd5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/1129855351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbb0f0 0x562c62dcdab0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 -- 172.21.15.106:0/1129855351 >> 172.21.15.106:0/1129855351 conn(0x562c62d4fd10 msgr2=0x562c62dbcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 -- 172.21.15.106:0/1129855351 shutdown_connections 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 -- 172.21.15.106:0/1129855351 wait complete. 2024-04-28T12:51:37.956 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.956+0000 7fe7ad76f740 1 -- start start 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7ad76f740 1 Processor -- start 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbb0f0 0x562c62dd0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dd1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7ad76f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 0x562c62dd4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562c62ddbdd0 con 0x562c62dbbae0 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c62ddb700 con 0x562c62dbb0f0 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7ad76f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562c62ddc4a0 con 0x562c62dc0c50 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7ad76f740 10 client.0 _ll_register_callbacks cb 0x7fe79000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7a2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dd1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbb0f0 0x562c62dd0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7a3400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbb0f0 0x562c62dd0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44888/0 (socket says 172.21.15.106:44888) 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7a3400640 1 -- 172.21.15.106:0/32054877 learned_addr learned my addr 172.21.15.106:0/32054877 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.957+0000 7fe7a2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dd1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45318/0 (socket says 172.21.15.106:45318) 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe7a3e00640 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 0x562c62dd4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe7a2a00640 1 -- 172.21.15.106:0/32054877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 msgr2=0x562c62dd4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.957 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 0x562c62dd4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe7a2a00640 1 -- 172.21.15.106:0/32054877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbb0f0 msgr2=0x562c62dd0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbb0f0 0x562c62dd0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe7a2a00640 1 -- 172.21.15.106:0/32054877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c62dd4920 con 0x562c62dbbae0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe7a2a00640 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dd1310 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c6a0 tx=0x7fe7900056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0baaaa87b4d0536 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe78d600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe78d600640 1 -- 172.21.15.106:0/32054877 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900bd0b0 con 0x562c62dbbae0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe78d600640 1 -- 172.21.15.106:0/32054877 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fe79009be80 con 0x562c62dbbae0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.958+0000 7fe7ad76f740 1 -- 172.21.15.106:0/32054877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe794040040 con 0x562c62dbbae0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe78d600640 1 -- 172.21.15.106:0/32054877 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe79009c570 con 0x562c62dbbae0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe7ad76f740 10 client.15285 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe7ad76f740 1 -- 172.21.15.106:0/32054877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c62e40e10 con 0x562c62dbbae0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe78d600640 1 -- 172.21.15.106:0/32054877 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe794040040 con 0x562c62dbbae0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe78d600640 1 -- 172.21.15.106:0/32054877 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fe7900870b0 con 0x562c62dbbae0 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe7ad76f740 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:37.958 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe7ad76f740 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-28T12:51:37.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe7ad76f740 20 client.15285 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:37.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x562c62e45880 0x562c62e44590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:37.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe7ad76f740 4 client.15285 mds_command: new command op to 24538 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:37.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe7ad76f740 1 -- 172.21.15.106:0/32054877 --> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] -- command(tid 0: {"prefix": "status"}) -- 0x562c62d27080 con 0x562c62e45880 2024-04-28T12:51:37.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.959+0000 7fe7a3400640 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x562c62e45880 0x562c62e44590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:37.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.960+0000 7fe7a3400640 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x562c62e45880 0x562c62e44590 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6fec5bb04d74613 server_cookie=87683c83fb1b6098 in_seq=0 out_seq=0 2024-04-28T12:51:37.959 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.960+0000 7fe78d600640 10 client.15285 ms_handle_connect on v2:172.21.15.171:6838/2406303763 2024-04-28T12:51:37.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.960+0000 7fe78d600640 1 -- 172.21.15.106:0/32054877 <== mds.3 v2:172.21.15.171:6838/2406303763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562c62d27080 con 0x562c62e45880 2024-04-28T12:51:37.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.960+0000 7fe78d600640 10 client.15285 handle_command_reply: tid=0 2024-04-28T12:51:37.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.960+0000 7fe7ad76f740 1 client.15285 shutdown 2024-04-28T12:51:37.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.960+0000 7fe7ad76f740 1 -- 172.21.15.106:0/32054877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 msgr2=0x562c62dd1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.960+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dd1310 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c6a0 tx=0x7fe7900056a0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.960 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe7ad76f740 1 -- 172.21.15.106:0/32054877 shutdown_connections 2024-04-28T12:51:37.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.171:6838/2406303763,v1:172.21.15.171:6839/2406303763] conn(0x562c62e45880 0x562c62e44590 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562c62dbbae0 0x562c62dd1310 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562c62dc0c50 0x562c62dd4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe7ad76f740 1 --2- 172.21.15.106:0/32054877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c62dbb0f0 0x562c62dd0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe78d600640 0 client.15285 ms_handle_reset on v2:172.21.15.171:6838/2406303763 2024-04-28T12:51:37.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe7ad76f740 1 -- 172.21.15.106:0/32054877 >> 172.21.15.106:0/32054877 conn(0x562c62d4fd10 msgr2=0x562c62dcdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe7ad76f740 1 -- 172.21.15.106:0/32054877 shutdown_connections 2024-04-28T12:51:37.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe7ad76f740 1 -- 172.21.15.106:0/32054877 wait complete. 2024-04-28T12:51:37.961 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.961+0000 7fe7ad76f740 20 client.15285 trim_cache size 0 max 16384 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 3, 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24538, 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 129.963794928, 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 180.07503764099999 2024-04-28T12:51:37.962 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:37.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 -- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7880a3cc0 msgr2=0x7fe7880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7880a3cc0 0x7fe7880a6180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x562c62dbc340 tx=0x7fe79c009500 comp rx=0 tx=0).stop 2024-04-28T12:51:37.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 -- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4105dc0 msgr2=0x7fe7a419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:37.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4105dc0 0x7fe7a419b3a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e9c0 tx=0x7fe79000ee80 comp rx=0 tx=0).stop 2024-04-28T12:51:37.963 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 -- 172.21.15.106:0/2266365540 shutdown_connections 2024-04-28T12:51:37.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe7880a3cc0 0x7fe7880a6180 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7a4105dc0 0x7fe7a419b3a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a41086f0 0x7fe7a41a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 --2- 172.21.15.106:0/2266365540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7a4103490 0x7fe7a419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:37.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 -- 172.21.15.106:0/2266365540 >> 172.21.15.106:0/2266365540 conn(0x7fe7a40fce80 msgr2=0x7fe7a4102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:37.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 -- 172.21.15.106:0/2266365540 shutdown_connections 2024-04-28T12:51:37.964 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:37.962+0000 7fe7aa800640 1 -- 172.21.15.106:0/2266365540 wait complete. 2024-04-28T12:51:37.974 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.g status 2024-04-28T12:51:38.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.165+0000 7fb775800640 1 -- 172.21.15.106:0/2444513207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb770105b10 msgr2=0x7fb770107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.165+0000 7fb775800640 1 --2- 172.21.15.106:0/2444513207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb770105b10 0x7fb770107f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb758009900 tx=0x7fb75802a410 comp rx=0 tx=0).stop 2024-04-28T12:51:38.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.165+0000 7fb775800640 1 -- 172.21.15.106:0/2444513207 shutdown_connections 2024-04-28T12:51:38.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.165+0000 7fb775800640 1 --2- 172.21.15.106:0/2444513207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb770108440 0x7fb77010a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.165+0000 7fb775800640 1 --2- 172.21.15.106:0/2444513207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb770105b10 0x7fb770107f00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.165+0000 7fb775800640 1 --2- 172.21.15.106:0/2444513207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7701031e0 0x7fb7701055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.165+0000 7fb775800640 1 -- 172.21.15.106:0/2444513207 >> 172.21.15.106:0/2444513207 conn(0x7fb7700fcbb0 msgr2=0x7fb7700ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.166+0000 7fb775800640 1 -- 172.21.15.106:0/2444513207 shutdown_connections 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.166+0000 7fb775800640 1 -- 172.21.15.106:0/2444513207 wait complete. 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.166+0000 7fb775800640 1 Processor -- start 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb775800640 1 -- start start 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb775800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7701031e0 0x7fb77019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb775800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb770105b10 0x7fb77019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.165 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb775800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb770108440 0x7fb7701a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb775800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010e2d0 con 0x7fb7701031e0 2024-04-28T12:51:38.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb775800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010e150 con 0x7fb770108440 2024-04-28T12:51:38.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb775800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb77010e450 con 0x7fb770105b10 2024-04-28T12:51:38.166 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7701031e0 0x7fb77019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7701031e0 0x7fb77019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45336/0 (socket says 172.21.15.106:45336) 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb76fe00640 1 -- 172.21.15.106:0/2353667206 learned_addr learned my addr 172.21.15.106:0/2353667206 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb774e00640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb770108440 0x7fb7701a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.167+0000 7fb767e00640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb770105b10 0x7fb77019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb76fe00640 1 -- 172.21.15.106:0/2353667206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb770105b10 msgr2=0x7fb77019b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb76fe00640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb770105b10 0x7fb77019b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb76fe00640 1 -- 172.21.15.106:0/2353667206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb770108440 msgr2=0x7fb7701a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb76fe00640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb770108440 0x7fb7701a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb76fe00640 1 -- 172.21.15.106:0/2353667206 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7701a27d0 con 0x7fb7701031e0 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb767e00640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb770105b10 0x7fb77019b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb774e00640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb770108440 0x7fb7701a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb76fe00640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7701031e0 0x7fb77019abb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e9f0 tx=0x7fb76000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb76d600640 1 -- 172.21.15.106:0/2353667206 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76000cd80 con 0x7fb7701031e0 2024-04-28T12:51:38.167 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb76d600640 1 -- 172.21.15.106:0/2353667206 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7600108e0 con 0x7fb7701031e0 2024-04-28T12:51:38.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb76d600640 1 -- 172.21.15.106:0/2353667206 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760018980 con 0x7fb7701031e0 2024-04-28T12:51:38.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.168+0000 7fb775800640 1 -- 172.21.15.106:0/2353667206 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7701a2ac0 con 0x7fb7701031e0 2024-04-28T12:51:38.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.169+0000 7fb775800640 1 -- 172.21.15.106:0/2353667206 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7701a2f90 con 0x7fb7701031e0 2024-04-28T12:51:38.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.173+0000 7fb779bcf740 1 Processor -- start 2024-04-28T12:51:38.168 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.173+0000 7fb779bcf740 1 -- start start 2024-04-28T12:51:38.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.173+0000 7fb76d600640 1 -- 172.21.15.106:0/2353667206 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb760010430 con 0x7fb7701031e0 2024-04-28T12:51:38.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.173+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e78f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.173+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e7947b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.173+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7972b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.173+0000 7fb779bcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56198e6f3600 con 0x56198e78f2c0 2024-04-28T12:51:38.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.173+0000 7fb779bcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56198e690100 con 0x56198e78fcb0 2024-04-28T12:51:38.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.173+0000 7fb779bcf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56198e6ef5c0 con 0x56198e794e20 2024-04-28T12:51:38.180 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb774e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e78f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb767e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb767e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49898/0 (socket says 172.21.15.106:49898) 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb767e00640 1 -- 172.21.15.106:0/168595250 learned_addr learned my addr 172.21.15.106:0/168595250 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb76fe00640 1 --2- 172.21.15.106:0/168595250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e7947b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb767e00640 1 -- 172.21.15.106:0/168595250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 msgr2=0x56198e7947b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb767e00640 1 --2- 172.21.15.106:0/168595250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e7947b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb767e00640 1 -- 172.21.15.106:0/168595250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 msgr2=0x56198e78f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb767e00640 1 --2- 172.21.15.106:0/168595250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e78f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb767e00640 1 -- 172.21.15.106:0/168595250 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56198e4052d0 con 0x56198e794e20 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb774e00640 1 --2- 172.21.15.106:0/168595250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e78f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.174+0000 7fb767e00640 1 --2- 172.21.15.106:0/168595250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7972b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb77006a620 tx=0x7fb758004140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fd002202337a12e server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.175+0000 7fb765600640 1 -- 172.21.15.106:0/168595250 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758035030 con 0x56198e794e20 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.175+0000 7fb765600640 1 -- 172.21.15.106:0/168595250 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7580054f0 con 0x56198e794e20 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.175+0000 7fb765600640 1 -- 172.21.15.106:0/168595250 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580035f0 con 0x56198e794e20 2024-04-28T12:51:38.181 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.175+0000 7fb76d600640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb7500a3cc0 0x7fb7500a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.175+0000 7fb76d600640 1 -- 172.21.15.106:0/2353667206 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fb7600d0d10 con 0x7fb7701031e0 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.175+0000 7fb774e00640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb7500a3cc0 0x7fb7500a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb779bcf740 1 -- 172.21.15.106:0/168595250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 msgr2=0x56198e7972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb779bcf740 1 --2- 172.21.15.106:0/168595250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7972b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb77006a620 tx=0x7fb758004140 comp rx=0 tx=0).stop 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb779bcf740 1 -- 172.21.15.106:0/168595250 shutdown_connections 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb779bcf740 1 --2- 172.21.15.106:0/168595250 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7972b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb779bcf740 1 --2- 172.21.15.106:0/168595250 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e7947b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb779bcf740 1 --2- 172.21.15.106:0/168595250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e78f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb779bcf740 1 -- 172.21.15.106:0/168595250 >> 172.21.15.106:0/168595250 conn(0x56198e723d10 msgr2=0x56198e78ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb774e00640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb7500a3cc0 0x7fb7500a6180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56198e790510 tx=0x7fb768009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb779bcf740 1 -- 172.21.15.106:0/168595250 shutdown_connections 2024-04-28T12:51:38.182 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.176+0000 7fb779bcf740 1 -- 172.21.15.106:0/168595250 wait complete. 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb779bcf740 1 -- start start 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb779bcf740 1 Processor -- start 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e799c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb779bcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56198e797d70 con 0x56198e78f2c0 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb779bcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56198e67af70 con 0x56198e78fcb0 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb779bcf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56198e797ef0 con 0x56198e794e20 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb779bcf740 10 client.0 _ll_register_callbacks cb 0x56198e6e9480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.177+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb774e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb767e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e799c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45360/0 (socket says 172.21.15.106:45360) 2024-04-28T12:51:38.183 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb76fe00640 1 -- 172.21.15.106:0/3619780264 learned_addr learned my addr 172.21.15.106:0/3619780264 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb774e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49900/0 (socket says 172.21.15.106:49900) 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb767e00640 1 -- 172.21.15.106:0/3619780264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 msgr2=0x56198e7a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb767e00640 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb767e00640 1 -- 172.21.15.106:0/3619780264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 msgr2=0x56198e799740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb767e00640 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb767e00640 1 -- 172.21.15.106:0/3619780264 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56198e4cf6b0 con 0x56198e78fcb0 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb767e00640 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e799c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb758006010 tx=0x7fb7580048d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0ef85be56c95ae server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.178+0000 7fb75e000640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.179+0000 7fb75e000640 1 -- 172.21.15.106:0/3619780264 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758035030 con 0x56198e78fcb0 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.179+0000 7fb779bcf740 1 -- 172.21.15.106:0/3619780264 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7580054f0 con 0x56198e78fcb0 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.179+0000 7fb75e000640 1 -- 172.21.15.106:0/3619780264 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fb758004fa0 con 0x56198e78fcb0 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.179+0000 7fb75e000640 1 -- 172.21.15.106:0/3619780264 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7580051a0 con 0x56198e78fcb0 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.179+0000 7fb779bcf740 10 client.25400 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.179+0000 7fb779bcf740 1 -- 172.21.15.106:0/3619780264 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56198e7a3ca0 con 0x56198e78fcb0 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.179+0000 7fb75e000640 1 -- 172.21.15.106:0/3619780264 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7580054f0 con 0x56198e78fcb0 2024-04-28T12:51:38.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.180+0000 7fb75e000640 1 -- 172.21.15.106:0/3619780264 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fb7580317a0 con 0x56198e78fcb0 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.180+0000 7fb779bcf740 10 client.25400 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.180+0000 7fb779bcf740 10 client.25400 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.180+0000 7fb779bcf740 20 client.25400 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.180+0000 7fb779bcf740 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x56198e7a5e60 0x56198e7a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.180+0000 7fb779bcf740 4 client.25400 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.180+0000 7fb779bcf740 1 -- 172.21.15.106:0/3619780264 --> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56198e701210 con 0x56198e7a5e60 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.180+0000 7fb76fe00640 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x56198e7a5e60 0x56198e7a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.181+0000 7fb76fe00640 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x56198e7a5e60 0x56198e7a72d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41e38a81fba7d9c3 server_cookie=f7e1354d6563e14d in_seq=0 out_seq=0 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.181+0000 7fb75e000640 10 client.25400 ms_handle_connect on v2:172.21.15.106:6838/3863798001 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.181+0000 7fb75e000640 1 -- 172.21.15.106:0/3619780264 <== mds.2 v2:172.21.15.106:6838/3863798001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56198e701210 con 0x56198e7a5e60 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.181+0000 7fb75e000640 10 client.25400 handle_command_reply: tid=0 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.181+0000 7fb779bcf740 1 client.25400 shutdown 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 -- 172.21.15.106:0/3619780264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 msgr2=0x56198e799c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.185 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e799c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb758006010 tx=0x7fb7580048d0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 -- 172.21.15.106:0/3619780264 shutdown_connections 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x56198e7a5e60 0x56198e7a72d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e7a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e799c80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb75e000640 0 client.25400 ms_handle_reset on v2:172.21.15.106:6838/3863798001 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 --2- 172.21.15.106:0/3619780264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 -- 172.21.15.106:0/3619780264 >> 172.21.15.106:0/3619780264 conn(0x56198e723d10 msgr2=0x56198e789f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 -- 172.21.15.106:0/3619780264 shutdown_connections 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 1 -- 172.21.15.106:0/3619780264 wait complete. 2024-04-28T12:51:38.186 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.182+0000 7fb779bcf740 20 client.25400 trim_cache size 0 max 16384 2024-04-28T12:51:38.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.218+0000 7fb779bcf740 1 Processor -- start 2024-04-28T12:51:38.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.218+0000 7fb779bcf740 1 -- start start 2024-04-28T12:51:38.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.218+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.218+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e78fcb0 0x56198e7896d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e794e20 0x56198e789c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb779bcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56198e7a5610 con 0x56198e78f2c0 2024-04-28T12:51:38.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb779bcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56198e7a5ce0 con 0x56198e794e20 2024-04-28T12:51:38.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb779bcf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56198e7a63b0 con 0x56198e78fcb0 2024-04-28T12:51:38.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45368/0 (socket says 172.21.15.106:45368) 2024-04-28T12:51:38.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb76fe00640 1 -- 172.21.15.106:0/689047195 learned_addr learned my addr 172.21.15.106:0/689047195 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb774e00640 1 --2- 172.21.15.106:0/689047195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e794e20 0x56198e789c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb767e00640 1 --2- 172.21.15.106:0/689047195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e78fcb0 0x56198e7896d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb76fe00640 1 -- 172.21.15.106:0/689047195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e78fcb0 msgr2=0x56198e7896d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb76fe00640 1 --2- 172.21.15.106:0/689047195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e78fcb0 0x56198e7896d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb76fe00640 1 -- 172.21.15.106:0/689047195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e794e20 msgr2=0x56198e789c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb76fe00640 1 --2- 172.21.15.106:0/689047195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e794e20 0x56198e789c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.219+0000 7fb76fe00640 1 -- 172.21.15.106:0/689047195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56198e786680 con 0x56198e78f2c0 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.220+0000 7fb76fe00640 1 --2- 172.21.15.106:0/689047195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799a60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb760099450 tx=0x7fb7600d47c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3decb2500789c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.220+0000 7fb764c00640 1 -- 172.21.15.106:0/689047195 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760010720 con 0x56198e78f2c0 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.220+0000 7fb764c00640 1 -- 172.21.15.106:0/689047195 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb76009d180 con 0x56198e78f2c0 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.220+0000 7fb764c00640 1 -- 172.21.15.106:0/689047195 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760010a90 con 0x56198e78f2c0 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 -- 172.21.15.106:0/689047195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 msgr2=0x56198e799a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 --2- 172.21.15.106:0/689047195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799a60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb760099450 tx=0x7fb7600d47c0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 -- 172.21.15.106:0/689047195 shutdown_connections 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 --2- 172.21.15.106:0/689047195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e794e20 0x56198e789c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 --2- 172.21.15.106:0/689047195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e78fcb0 0x56198e7896d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 --2- 172.21.15.106:0/689047195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e799a60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 -- 172.21.15.106:0/689047195 >> 172.21.15.106:0/689047195 conn(0x56198e723d10 msgr2=0x56198e799350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 -- 172.21.15.106:0/689047195 shutdown_connections 2024-04-28T12:51:38.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 -- 172.21.15.106:0/689047195 wait complete. 2024-04-28T12:51:38.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 -- start start 2024-04-28T12:51:38.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.221+0000 7fb779bcf740 1 Processor -- start 2024-04-28T12:51:38.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e79c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e79cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb779bcf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e79fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb779bcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56198e7a7150 con 0x56198e78f2c0 2024-04-28T12:51:38.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb779bcf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56198e7a6a80 con 0x56198e78fcb0 2024-04-28T12:51:38.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb779bcf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56198e7a7820 con 0x56198e794e20 2024-04-28T12:51:38.223 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb779bcf740 10 client.0 _ll_register_callbacks cb 0x7fb768009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e79c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e79c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45374/0 (socket says 172.21.15.106:45374) 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb76fe00640 1 -- 172.21.15.106:0/16175456 learned_addr learned my addr 172.21.15.106:0/16175456 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb767e00640 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e79cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb774e00640 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e79fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb76fe00640 1 -- 172.21.15.106:0/16175456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 msgr2=0x56198e79fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb76fe00640 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e79fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb76fe00640 1 -- 172.21.15.106:0/16175456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 msgr2=0x56198e79cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb76fe00640 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e79cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.222+0000 7fb76fe00640 1 -- 172.21.15.106:0/16175456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56198e7a0140 con 0x56198e78f2c0 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb76fe00640 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e79c640 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb7600bd010 tx=0x7fb7600bd1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37883f72c60e364 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb75e000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb75e000640 1 -- 172.21.15.106:0/16175456 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76009ca80 con 0x56198e78f2c0 2024-04-28T12:51:38.224 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb75e000640 1 -- 172.21.15.106:0/16175456 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fb7600990f0 con 0x56198e78f2c0 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb779bcf740 1 -- 172.21.15.106:0/16175456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb76009d180 con 0x56198e78f2c0 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb75e000640 1 -- 172.21.15.106:0/16175456 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7600a0e30 con 0x56198e78f2c0 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb779bcf740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb779bcf740 1 -- 172.21.15.106:0/16175456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56198e80c240 con 0x56198e78f2c0 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb75e000640 1 -- 172.21.15.106:0/16175456 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb76009d180 con 0x56198e78f2c0 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.223+0000 7fb75e000640 1 -- 172.21.15.106:0/16175456 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fb7600d5040 con 0x56198e78f2c0 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.224+0000 7fb779bcf740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.224+0000 7fb779bcf740 10 client.15303 resolve_mds: resolved name 'g' to daemon mds.g 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.224+0000 7fb779bcf740 20 client.15303 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.224+0000 7fb779bcf740 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x56198e8108d0 0x56198e80f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.224+0000 7fb779bcf740 4 client.15303 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.224+0000 7fb779bcf740 1 -- 172.21.15.106:0/16175456 --> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] -- command(tid 0: {"prefix": "status"}) -- 0x56198e6fb080 con 0x56198e8108d0 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.224+0000 7fb767e00640 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x56198e8108d0 0x56198e80f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.225 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.224+0000 7fb75e000640 10 client.15303 ms_handle_connect on v2:172.21.15.106:6838/3863798001 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.224+0000 7fb767e00640 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x56198e8108d0 0x56198e80f5e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e46b628dfa7ce37c server_cookie=275b920ebc364f5c in_seq=0 out_seq=0 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb75e000640 1 -- 172.21.15.106:0/16175456 <== mds.2 v2:172.21.15.106:6838/3863798001 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56198e6fb080 con 0x56198e8108d0 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb75e000640 10 client.15303 handle_command_reply: tid=0 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 client.15303 shutdown 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 -- 172.21.15.106:0/16175456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 msgr2=0x56198e79c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e79c640 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb7600bd010 tx=0x7fb7600bd1b0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 -- 172.21.15.106:0/16175456 shutdown_connections 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.106:6838/3863798001,v1:172.21.15.106:6839/3863798001] conn(0x56198e8108d0 0x56198e80f5e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56198e794e20 0x56198e79fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.226 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56198e78fcb0 0x56198e79cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb75e000640 0 client.15303 ms_handle_reset on v2:172.21.15.106:6838/3863798001 2024-04-28T12:51:38.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 --2- 172.21.15.106:0/16175456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56198e78f2c0 0x56198e79c640 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 -- 172.21.15.106:0/16175456 >> 172.21.15.106:0/16175456 conn(0x56198e723d10 msgr2=0x56198e7896d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 -- 172.21.15.106:0/16175456 shutdown_connections 2024-04-28T12:51:38.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.225+0000 7fb779bcf740 1 -- 172.21.15.106:0/16175456 wait complete. 2024-04-28T12:51:38.227 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb779bcf740 20 client.15303 trim_cache size 0 max 16384 2024-04-28T12:51:38.227 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:38.227 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:38.227 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 2, 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24548, 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 131.24133867399999, 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 176.44049190000001 2024-04-28T12:51:38.228 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:38.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 -- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb7500a3cc0 msgr2=0x7fb7500a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb7500a3cc0 0x7fb7500a6180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56198e790510 tx=0x7fb768009500 comp rx=0 tx=0).stop 2024-04-28T12:51:38.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 -- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7701031e0 msgr2=0x7fb77019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7701031e0 0x7fb77019abb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e9f0 tx=0x7fb76000eeb0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 -- 172.21.15.106:0/2353667206 shutdown_connections 2024-04-28T12:51:38.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fb7500a3cc0 0x7fb7500a6180 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb770108440 0x7fb7701a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.229 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb770105b10 0x7fb77019b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 --2- 172.21.15.106:0/2353667206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7701031e0 0x7fb77019abb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.226+0000 7fb775800640 1 -- 172.21.15.106:0/2353667206 >> 172.21.15.106:0/2353667206 conn(0x7fb7700fcbb0 msgr2=0x7fb770102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.227+0000 7fb775800640 1 -- 172.21.15.106:0/2353667206 shutdown_connections 2024-04-28T12:51:38.230 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.227+0000 7fb775800640 1 -- 172.21.15.106:0/2353667206 wait complete. 2024-04-28T12:51:38.246 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-28T12:51:38.306 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:38 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1655723205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:38.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.438+0000 7ff8b7200640 1 -- 172.21.15.106:0/1582021681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b8075390 msgr2=0x7ff8b8075810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.435 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.438+0000 7ff8b7200640 1 --2- 172.21.15.106:0/1582021681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b8075390 0x7ff8b8075810 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009930 tx=0x7ff8a002a420 comp rx=0 tx=0).stop 2024-04-28T12:51:38.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.438+0000 7ff8b7200640 1 -- 172.21.15.106:0/1582021681 shutdown_connections 2024-04-28T12:51:38.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.438+0000 7ff8b7200640 1 --2- 172.21.15.106:0/1582021681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b8075390 0x7ff8b8075810 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.438+0000 7ff8b7200640 1 --2- 172.21.15.106:0/1582021681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8b810ed30 0x7ff8b8111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.438+0000 7ff8b7200640 1 --2- 172.21.15.106:0/1582021681 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8b8076970 0x7ff8b8074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.438+0000 7ff8b7200640 1 -- 172.21.15.106:0/1582021681 >> 172.21.15.106:0/1582021681 conn(0x7ff8b80fdcb0 msgr2=0x7ff8b8100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.438+0000 7ff8b7200640 1 -- 172.21.15.106:0/1582021681 shutdown_connections 2024-04-28T12:51:38.436 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.438+0000 7ff8b7200640 1 -- 172.21.15.106:0/1582021681 wait complete. 2024-04-28T12:51:38.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.439+0000 7ff8b7200640 1 Processor -- start 2024-04-28T12:51:38.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.439+0000 7ff8b7200640 1 -- start start 2024-04-28T12:51:38.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.439+0000 7ff8b7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8b8075390 0x7ff8b819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.439+0000 7ff8b7200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8b8076970 0x7ff8b819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.439+0000 7ff8b7200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b810ed30 0x7ff8b81a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.439+0000 7ff8b7200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b8114860 con 0x7ff8b810ed30 2024-04-28T12:51:38.439 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.439+0000 7ff8b7200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b81146e0 con 0x7ff8b8076970 2024-04-28T12:51:38.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.439+0000 7ff8b7200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b81149e0 con 0x7ff8b8075390 2024-04-28T12:51:38.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8b6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b810ed30 0x7ff8b81a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8b8076970 0x7ff8b819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8b5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8b8075390 0x7ff8b819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8b8076970 0x7ff8b819f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44964/0 (socket says 172.21.15.106:44964) 2024-04-28T12:51:38.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8afe00640 1 -- 172.21.15.106:0/2509686594 learned_addr learned my addr 172.21.15.106:0/2509686594 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.440 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8b6800640 1 -- 172.21.15.106:0/2509686594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8b8075390 msgr2=0x7ff8b819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8b6800640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8b8075390 0x7ff8b819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8b6800640 1 -- 172.21.15.106:0/2509686594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8b8076970 msgr2=0x7ff8b819f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8b6800640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8b8076970 0x7ff8b819f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8b6800640 1 -- 172.21.15.106:0/2509686594 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b81a6bf0 con 0x7ff8b810ed30 2024-04-28T12:51:38.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8b8075390 0x7ff8b819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:38.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.440+0000 7ff8afe00640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8b8076970 0x7ff8b819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:38.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.441+0000 7ff8b6800640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b810ed30 0x7ff8b81a6580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009fd0 tx=0x7ff8b000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.441+0000 7ff8aea00640 1 -- 172.21.15.106:0/2509686594 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b000ce20 con 0x7ff8b810ed30 2024-04-28T12:51:38.441 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.441+0000 7ff8aea00640 1 -- 172.21.15.106:0/2509686594 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b0005020 con 0x7ff8b810ed30 2024-04-28T12:51:38.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.441+0000 7ff8b7200640 1 -- 172.21.15.106:0/2509686594 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b81a6ee0 con 0x7ff8b810ed30 2024-04-28T12:51:38.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.441+0000 7ff8aea00640 1 -- 172.21.15.106:0/2509686594 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0010690 con 0x7ff8b810ed30 2024-04-28T12:51:38.442 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.441+0000 7ff8b7200640 1 -- 172.21.15.106:0/2509686594 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b81a73a0 con 0x7ff8b810ed30 2024-04-28T12:51:38.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.445+0000 7ff8c0068740 1 Processor -- start 2024-04-28T12:51:38.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.445+0000 7ff8aea00640 1 -- 172.21.15.106:0/2509686594 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff8b00054d0 con 0x7ff8b810ed30 2024-04-28T12:51:38.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.445+0000 7ff8c0068740 1 -- start start 2024-04-28T12:51:38.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e208202c0 0x556e208206e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20820cb0 0x556e208257b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e208282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8c0068740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e20784600 con 0x556e20825e20 2024-04-28T12:51:38.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8c0068740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556e20721100 con 0x556e208202c0 2024-04-28T12:51:38.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8c0068740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556e207805c0 con 0x556e20820cb0 2024-04-28T12:51:38.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e208202c0 0x556e208206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e208282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8b5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20820cb0 0x556e208257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8b5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20820cb0 0x556e208257b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:49944/0 (socket says 172.21.15.106:49944) 2024-04-28T12:51:38.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.446+0000 7ff8b5e00640 1 -- 172.21.15.106:0/2981988387 learned_addr learned my addr 172.21.15.106:0/2981988387 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8b5e00640 1 -- 172.21.15.106:0/2981988387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e208202c0 msgr2=0x556e208206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/2981988387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e208202c0 0x556e208206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8b5e00640 1 -- 172.21.15.106:0/2981988387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 msgr2=0x556e208282b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/2981988387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e208282b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8b5e00640 1 -- 172.21.15.106:0/2981988387 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e204962d0 con 0x556e20820cb0 2024-04-28T12:51:38.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8aea00640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff89c0a3cc0 0x7ff89c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/2981988387 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20820cb0 0x556e208257b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8010580 tx=0x7ff8a8010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=816b2ac7bfe0c476 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8aea00640 1 -- 172.21.15.106:0/2509686594 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7ff8b00d0f60 con 0x7ff8b810ed30 2024-04-28T12:51:38.446 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8afe00640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff89c0a3cc0 0x7ff89c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8a7400640 1 -- 172.21.15.106:0/2981988387 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8011760 con 0x556e20820cb0 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8a7400640 1 -- 172.21.15.106:0/2981988387 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a8011db0 con 0x556e20820cb0 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.447+0000 7ff8a7400640 1 -- 172.21.15.106:0/2981988387 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800ed90 con 0x556e20820cb0 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.448+0000 7ff8afe00640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff89c0a3cc0 0x7ff89c0a6180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8b81a04e0 tx=0x7ff8a002a420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.448+0000 7ff8c0068740 1 -- 172.21.15.106:0/2981988387 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20820cb0 msgr2=0x556e208257b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.448+0000 7ff8c0068740 1 --2- 172.21.15.106:0/2981988387 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20820cb0 0x556e208257b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8010580 tx=0x7ff8a8010a40 comp rx=0 tx=0).stop 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.449+0000 7ff8c0068740 1 -- 172.21.15.106:0/2981988387 shutdown_connections 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.449+0000 7ff8c0068740 1 --2- 172.21.15.106:0/2981988387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e208282b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.449+0000 7ff8c0068740 1 --2- 172.21.15.106:0/2981988387 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20820cb0 0x556e208257b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.449+0000 7ff8c0068740 1 --2- 172.21.15.106:0/2981988387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e208202c0 0x556e208206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.447 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.449+0000 7ff8c0068740 1 -- 172.21.15.106:0/2981988387 >> 172.21.15.106:0/2981988387 conn(0x556e207b4d10 msgr2=0x556e2081bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.449+0000 7ff8c0068740 1 -- 172.21.15.106:0/2981988387 shutdown_connections 2024-04-28T12:51:38.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.449+0000 7ff8c0068740 1 -- 172.21.15.106:0/2981988387 wait complete. 2024-04-28T12:51:38.448 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.449+0000 7ff8c0068740 1 -- start start 2024-04-28T12:51:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.449+0000 7ff8c0068740 1 Processor -- start 2024-04-28T12:51:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e208202c0 0x556e2082a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 0x556e2082acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20825e20 0x556e20831c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8c0068740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e20828dd0 con 0x556e208202c0 2024-04-28T12:51:38.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8c0068740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556e2070bf70 con 0x556e20820cb0 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8c0068740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556e20828f50 con 0x556e20825e20 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8c0068740 10 client.0 _ll_register_callbacks cb 0x556e2077a480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8b5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e208202c0 0x556e2082a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8b6800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20825e20 0x556e20831c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8b5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e208202c0 0x556e2082a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45416/0 (socket says 172.21.15.106:45416) 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8b5e00640 1 -- 172.21.15.106:0/1303313409 learned_addr learned my addr 172.21.15.106:0/1303313409 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.450+0000 7ff8afe00640 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 0x556e2082acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff8b5e00640 1 -- 172.21.15.106:0/1303313409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20825e20 msgr2=0x556e20831c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20825e20 0x556e20831c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff8b5e00640 1 -- 172.21.15.106:0/1303313409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 msgr2=0x556e2082acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 0x556e2082acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff8b5e00640 1 -- 172.21.15.106:0/1303313409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e205606b0 con 0x556e208202c0 2024-04-28T12:51:38.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff8afe00640 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 0x556e2082acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff8b6800640 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20825e20 0x556e20831c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e208202c0 0x556e2082a770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8005d00 tx=0x7ff8a800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e482cfe6fe40040 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff89be00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff89be00640 1 -- 172.21.15.106:0/1303313409 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a802b060 con 0x556e208202c0 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff89be00640 1 -- 172.21.15.106:0/1303313409 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7ff8a800ed20 con 0x556e208202c0 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.451+0000 7ff8c0068740 1 -- 172.21.15.106:0/1303313409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a8011db0 con 0x556e208202c0 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff89be00640 1 -- 172.21.15.106:0/1303313409 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff8a8032050 con 0x556e208202c0 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff8c0068740 10 client.15315 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff8c0068740 1 -- 172.21.15.106:0/1303313409 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e20834d00 con 0x556e208202c0 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff89be00640 1 -- 172.21.15.106:0/1303313409 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a8011db0 con 0x556e208202c0 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff89be00640 1 -- 172.21.15.106:0/1303313409 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ff8a801c6e0 con 0x556e208202c0 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff8c0068740 10 client.15315 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff8c0068740 10 client.15315 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-28T12:51:38.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff8c0068740 20 client.15315 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x556e20837000 0x556e20838470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff8c0068740 4 client.15315 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.452+0000 7ff8c0068740 1 -- 172.21.15.106:0/1303313409 --> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e20792210 con 0x556e20837000 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.453+0000 7ff8afe00640 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x556e20837000 0x556e20838470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.453+0000 7ff8afe00640 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x556e20837000 0x556e20838470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98db811902858490 server_cookie=846432a30ae25f76 in_seq=0 out_seq=0 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.453+0000 7ff89be00640 10 client.15315 ms_handle_connect on v2:172.21.15.106:6840/92504617 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff89be00640 1 -- 172.21.15.106:0/1303313409 <== mds.1 v2:172.21.15.106:6840/92504617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e20792210 con 0x556e20837000 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff89be00640 10 client.15315 handle_command_reply: tid=0 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff8c0068740 1 client.15315 shutdown 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff8c0068740 1 -- 172.21.15.106:0/1303313409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e208202c0 msgr2=0x556e2082a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e208202c0 0x556e2082a770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8005d00 tx=0x7ff8a800be70 comp rx=0 tx=0).stop 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff8c0068740 1 -- 172.21.15.106:0/1303313409 shutdown_connections 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x556e20837000 0x556e20838470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e20825e20 0x556e20831c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff89be00640 0 client.15315 ms_handle_reset on v2:172.21.15.106:6840/92504617 2024-04-28T12:51:38.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 0x556e2082acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1303313409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e208202c0 0x556e2082a770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.454+0000 7ff8c0068740 1 -- 172.21.15.106:0/1303313409 >> 172.21.15.106:0/1303313409 conn(0x556e207b4d10 msgr2=0x556e2081af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.455+0000 7ff8c0068740 1 -- 172.21.15.106:0/1303313409 shutdown_connections 2024-04-28T12:51:38.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.455+0000 7ff8c0068740 1 -- 172.21.15.106:0/1303313409 wait complete. 2024-04-28T12:51:38.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.455+0000 7ff8c0068740 20 client.15315 trim_cache size 0 max 16384 2024-04-28T12:51:38.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.474+0000 7ff8c0068740 1 Processor -- start 2024-04-28T12:51:38.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.474+0000 7ff8c0068740 1 -- start start 2024-04-28T12:51:38.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.474+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 0x556e20838470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.474+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20820cb0 0x556e20831ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.474+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20825e20 0x556e2081ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.474+0000 7ff8c0068740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e208367b0 con 0x556e20820cb0 2024-04-28T12:51:38.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.474+0000 7ff8c0068740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556e20836e80 con 0x556e20825e20 2024-04-28T12:51:38.472 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.474+0000 7ff8c0068740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556e20837550 con 0x556e208202c0 2024-04-28T12:51:38.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20820cb0 0x556e20831ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8b6800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20825e20 0x556e2081ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8afe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20820cb0 0x556e20831ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45422/0 (socket says 172.21.15.106:45422) 2024-04-28T12:51:38.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8b6800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20825e20 0x556e2081ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:44988/0 (socket says 172.21.15.106:44988) 2024-04-28T12:51:38.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8afe00640 1 -- 172.21.15.106:0/1230393889 learned_addr learned my addr 172.21.15.106:0/1230393889 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8b5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 0x556e20838470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.473 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8b5e00640 1 -- 172.21.15.106:0/1230393889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20825e20 msgr2=0x556e2081ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/1230393889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20825e20 0x556e2081ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8b5e00640 1 -- 172.21.15.106:0/1230393889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20820cb0 msgr2=0x556e20831ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/1230393889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20820cb0 0x556e20831ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.475+0000 7ff8b5e00640 1 -- 172.21.15.106:0/1230393889 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e20817460 con 0x556e208202c0 2024-04-28T12:51:38.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.476+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/1230393889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 0x556e20838470 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8010a80 tx=0x7ff8a8010fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c54da473c3b24e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.476+0000 7ff8a6a00640 1 -- 172.21.15.106:0/1230393889 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a802b040 con 0x556e208202c0 2024-04-28T12:51:38.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.476+0000 7ff8a6a00640 1 -- 172.21.15.106:0/1230393889 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a800ed20 con 0x556e208202c0 2024-04-28T12:51:38.474 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.476+0000 7ff8a6a00640 1 -- 172.21.15.106:0/1230393889 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a801c6e0 con 0x556e208202c0 2024-04-28T12:51:38.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 -- 172.21.15.106:0/1230393889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 msgr2=0x556e20838470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1230393889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 0x556e20838470 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8010a80 tx=0x7ff8a8010fc0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.477 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 -- 172.21.15.106:0/1230393889 shutdown_connections 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1230393889 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20820cb0 0x556e20831ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1230393889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20825e20 0x556e2081ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 --2- 172.21.15.106:0/1230393889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 0x556e20838470 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 -- 172.21.15.106:0/1230393889 >> 172.21.15.106:0/1230393889 conn(0x556e207b4d10 msgr2=0x556e2081a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 -- 172.21.15.106:0/1230393889 shutdown_connections 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 -- 172.21.15.106:0/1230393889 wait complete. 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 -- start start 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.477+0000 7ff8c0068740 1 Processor -- start 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 0x556e208314f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 0x556e20831a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8c0068740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e2082df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8c0068740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e208382f0 con 0x556e20825e20 2024-04-28T12:51:38.478 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8c0068740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556e20837c20 con 0x556e20820cb0 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8c0068740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556e208389c0 con 0x556e208202c0 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8c0068740 10 client.0 _ll_register_callbacks cb 0x7ff8a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8b6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e2082df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8b5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 0x556e208314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8b6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e2082df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45426/0 (socket says 172.21.15.106:45426) 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8b6800640 1 -- 172.21.15.106:0/743395364 learned_addr learned my addr 172.21.15.106:0/743395364 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8afe00640 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 0x556e20831a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8b6800640 1 -- 172.21.15.106:0/743395364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 msgr2=0x556e208314f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8b6800640 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 0x556e208314f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8b6800640 1 -- 172.21.15.106:0/743395364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 msgr2=0x556e20831a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.478+0000 7ff8b6800640 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 0x556e20831a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.479+0000 7ff8b6800640 1 -- 172.21.15.106:0/743395364 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e2082e7c0 con 0x556e20825e20 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.479+0000 7ff8b6800640 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e2082df90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00ce530 tx=0x7ff8b009cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d014dffcd96f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.479+0000 7ff89be00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.479+0000 7ff89be00640 1 -- 172.21.15.106:0/743395364 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00bb060 con 0x556e20825e20 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.479+0000 7ff89be00640 1 -- 172.21.15.106:0/743395364 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7ff8b00bd0b0 con 0x556e20825e20 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.479+0000 7ff8c0068740 1 -- 172.21.15.106:0/743395364 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a800ed20 con 0x556e20825e20 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.479+0000 7ff89be00640 1 -- 172.21.15.106:0/743395364 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff8b0082050 con 0x556e20825e20 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.479+0000 7ff8c0068740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.479+0000 7ff8c0068740 1 -- 172.21.15.106:0/743395364 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e2089dd70 con 0x556e20825e20 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.480+0000 7ff89be00640 1 -- 172.21.15.106:0/743395364 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a800ed20 con 0x556e20825e20 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.480+0000 7ff89be00640 1 -- 172.21.15.106:0/743395364 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7ff8b00ceb10 con 0x556e20825e20 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.480+0000 7ff8c0068740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:38.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.480+0000 7ff8c0068740 10 client.15321 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-28T12:51:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.480+0000 7ff8c0068740 20 client.15321 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.480+0000 7ff8c0068740 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x556e208a2910 0x556e2089ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.480+0000 7ff8c0068740 4 client.15321 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.480+0000 7ff8c0068740 1 -- 172.21.15.106:0/743395364 --> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] -- command(tid 0: {"prefix": "status"}) -- 0x556e2078c080 con 0x556e208a2910 2024-04-28T12:51:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.480+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x556e208a2910 0x556e2089ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff89be00640 10 client.15321 ms_handle_connect on v2:172.21.15.106:6840/92504617 2024-04-28T12:51:38.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8b5e00640 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x556e208a2910 0x556e2089ed50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e39e7d4965f04db7 server_cookie=d4056d21106cd882 in_seq=0 out_seq=0 2024-04-28T12:51:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff89be00640 1 -- 172.21.15.106:0/743395364 <== mds.1 v2:172.21.15.106:6840/92504617 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556e2078c080 con 0x556e208a2910 2024-04-28T12:51:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff89be00640 10 client.15321 handle_command_reply: tid=0 2024-04-28T12:51:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8c0068740 1 client.15321 shutdown 2024-04-28T12:51:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8c0068740 1 -- 172.21.15.106:0/743395364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 msgr2=0x556e2082df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8c0068740 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e2082df90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff8b00ce530 tx=0x7ff8b009cd90 comp rx=0 tx=0).stop 2024-04-28T12:51:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8c0068740 1 -- 172.21.15.106:0/743395364 shutdown_connections 2024-04-28T12:51:38.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8c0068740 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.106:6840/92504617,v1:172.21.15.106:6841/92504617] conn(0x556e208a2910 0x556e2089ed50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8c0068740 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e20825e20 0x556e2082df90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8c0068740 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556e20820cb0 0x556e20831a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff89be00640 0 client.15321 ms_handle_reset on v2:172.21.15.106:6840/92504617 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8c0068740 1 --2- 172.21.15.106:0/743395364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556e208202c0 0x556e208314f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.481+0000 7ff8c0068740 1 -- 172.21.15.106:0/743395364 >> 172.21.15.106:0/743395364 conn(0x556e207b4d10 msgr2=0x556e2081af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8c0068740 1 -- 172.21.15.106:0/743395364 shutdown_connections 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8c0068740 1 -- 172.21.15.106:0/743395364 wait complete. 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8c0068740 20 client.15321 trim_cache size 0 max 16384 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:38.483 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 1, 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24566, 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 132.51849456799999, 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 172.59095574299999 2024-04-28T12:51:38.484 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 -- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff89c0a3cc0 msgr2=0x7ff89c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff89c0a3cc0 0x7ff89c0a6180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff8b81a04e0 tx=0x7ff8a002a420 comp rx=0 tx=0).stop 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 -- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b810ed30 msgr2=0x7ff8b81a6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b810ed30 0x7ff8b81a6580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009fd0 tx=0x7ff8b000ef10 comp rx=0 tx=0).stop 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 -- 172.21.15.106:0/2509686594 shutdown_connections 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff89c0a3cc0 0x7ff89c0a6180 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8b810ed30 0x7ff8b81a6580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8b8076970 0x7ff8b819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 --2- 172.21.15.106:0/2509686594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8b8075390 0x7ff8b819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.485 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.482+0000 7ff8b7200640 1 -- 172.21.15.106:0/2509686594 >> 172.21.15.106:0/2509686594 conn(0x7ff8b80fdcb0 msgr2=0x7ff8b81020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.483+0000 7ff8b7200640 1 -- 172.21.15.106:0/2509686594 shutdown_connections 2024-04-28T12:51:38.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.483+0000 7ff8b7200640 1 -- 172.21.15.106:0/2509686594 wait complete. 2024-04-28T12:51:38.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:38 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1655723205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:38.498 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-28T12:51:38.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:38 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1655723205' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T12:51:38.686 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.689+0000 7f1b65e00640 1 -- 172.21.15.106:0/1949395912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 msgr2=0x7f1b60101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.689+0000 7f1b65e00640 1 --2- 172.21.15.106:0/1949395912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 0x7f1b60101db0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009900 tx=0x7f1b4802a3f0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.689+0000 7f1b65e00640 1 -- 172.21.15.106:0/1949395912 shutdown_connections 2024-04-28T12:51:38.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.689+0000 7f1b65e00640 1 --2- 172.21.15.106:0/1949395912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 0x7f1b60101db0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.689+0000 7f1b65e00640 1 --2- 172.21.15.106:0/1949395912 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b601022f0 0x7f1b60102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.689+0000 7f1b65e00640 1 --2- 172.21.15.106:0/1949395912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b601008e0 0x7f1b60100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.689+0000 7f1b65e00640 1 -- 172.21.15.106:0/1949395912 >> 172.21.15.106:0/1949395912 conn(0x7f1b600fc120 msgr2=0x7f1b600fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.689+0000 7f1b65e00640 1 -- 172.21.15.106:0/1949395912 shutdown_connections 2024-04-28T12:51:38.687 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.689+0000 7f1b65e00640 1 -- 172.21.15.106:0/1949395912 wait complete. 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.690+0000 7f1b65e00640 1 Processor -- start 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.690+0000 7f1b65e00640 1 -- start start 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.690+0000 7f1b65e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b601008e0 0x7f1b6019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.690+0000 7f1b65e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 0x7f1b6019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.690+0000 7f1b65e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b601022f0 0x7f1b601a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.690+0000 7f1b65e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60110700 con 0x7f1b60101950 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.690+0000 7f1b65e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60110580 con 0x7f1b601008e0 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.690+0000 7f1b65e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1b60110880 con 0x7f1b601022f0 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b56000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 0x7f1b6019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b56000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 0x7f1b6019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45442/0 (socket says 172.21.15.106:45442) 2024-04-28T12:51:38.689 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b56000640 1 -- 172.21.15.106:0/3423016203 learned_addr learned my addr 172.21.15.106:0/3423016203 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b5ea00640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b601008e0 0x7f1b6019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b5f400640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b601022f0 0x7f1b601a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b56000640 1 -- 172.21.15.106:0/3423016203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b601022f0 msgr2=0x7f1b601a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b56000640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b601022f0 0x7f1b601a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b56000640 1 -- 172.21.15.106:0/3423016203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b601008e0 msgr2=0x7f1b6019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b56000640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b601008e0 0x7f1b6019ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b56000640 1 -- 172.21.15.106:0/3423016203 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b601a2a30 con 0x7f1b60101950 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.691+0000 7f1b56000640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 0x7f1b6019b350 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1b48005ab0 tx=0x7f1b48003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.692+0000 7f1b64a00640 1 -- 172.21.15.106:0/3423016203 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4803f070 con 0x7f1b60101950 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.692+0000 7f1b64a00640 1 -- 172.21.15.106:0/3423016203 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b48031c50 con 0x7f1b60101950 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.692+0000 7f1b65e00640 1 -- 172.21.15.106:0/3423016203 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b601a2cc0 con 0x7f1b60101950 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.692+0000 7f1b64a00640 1 -- 172.21.15.106:0/3423016203 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48035030 con 0x7f1b60101950 2024-04-28T12:51:38.690 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.692+0000 7f1b65e00640 1 -- 172.21.15.106:0/3423016203 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b601a31b0 con 0x7f1b60101950 2024-04-28T12:51:38.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.695+0000 7f1b64a00640 1 -- 172.21.15.106:0/3423016203 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f1b48042070 con 0x7f1b60101950 2024-04-28T12:51:38.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.695+0000 7f1b68c2b740 1 Processor -- start 2024-04-28T12:51:38.695 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.695+0000 7f1b68c2b740 1 -- start start 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.695+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x55594565e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.695+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 0x5559456635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.695+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 0x5559456660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.695+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5559455c2600 con 0x55594565e0f0 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.695+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55594555f100 con 0x55594565eae0 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.695+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5559455be5c0 con 0x555945663c50 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x55594565e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b56000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 0x5559456660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x55594565e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45458/0 (socket says 172.21.15.106:45458) 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b5f400640 1 -- 172.21.15.106:0/1615367193 learned_addr learned my addr 172.21.15.106:0/1615367193 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b5ea00640 1 --2- 172.21.15.106:0/1615367193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 0x5559456635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b56000640 1 -- 172.21.15.106:0/1615367193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 msgr2=0x5559456635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b56000640 1 --2- 172.21.15.106:0/1615367193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 0x5559456635e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.696 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b56000640 1 -- 172.21.15.106:0/1615367193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 msgr2=0x55594565e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b56000640 1 --2- 172.21.15.106:0/1615367193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x55594565e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b56000640 1 -- 172.21.15.106:0/1615367193 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559452d42d0 con 0x555945663c50 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.696+0000 7f1b56000640 1 --2- 172.21.15.106:0/1615367193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 0x5559456660e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1b480ee5b0 tx=0x7f1b480f92d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27357c1533f54896 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.697+0000 7f1b64a00640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1b440a3c40 0x7f1b440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.697+0000 7f1b64a00640 1 -- 172.21.15.106:0/3423016203 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f1b480f0ff0 con 0x7f1b60101950 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.697+0000 7f1b4fe00640 1 -- 172.21.15.106:0/1615367193 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480f32e0 con 0x555945663c50 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.697+0000 7f1b5f400640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1b440a3c40 0x7f1b440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.697+0000 7f1b4fe00640 1 -- 172.21.15.106:0/1615367193 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b480fe030 con 0x555945663c50 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.697+0000 7f1b4fe00640 1 -- 172.21.15.106:0/1615367193 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480bc790 con 0x555945663c50 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 -- 172.21.15.106:0/1615367193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 msgr2=0x5559456660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b5f400640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1b440a3c40 0x7f1b440a6100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55594565f340 tx=0x7f1b58009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/1615367193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 0x5559456660e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1b480ee5b0 tx=0x7f1b480f92d0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 -- 172.21.15.106:0/1615367193 shutdown_connections 2024-04-28T12:51:38.697 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/1615367193 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 0x5559456660e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/1615367193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 0x5559456635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/1615367193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x55594565e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 -- 172.21.15.106:0/1615367193 >> 172.21.15.106:0/1615367193 conn(0x5559455f2d10 msgr2=0x555945659b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 -- 172.21.15.106:0/1615367193 shutdown_connections 2024-04-28T12:51:38.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 -- 172.21.15.106:0/1615367193 wait complete. 2024-04-28T12:51:38.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.698+0000 7f1b68c2b740 1 -- start start 2024-04-28T12:51:38.698 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b68c2b740 1 Processor -- start 2024-04-28T12:51:38.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x555945668260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565eae0 0x5559456687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555945663c50 0x55594566f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5559456669c0 con 0x55594565e0f0 2024-04-28T12:51:38.702 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555945549f70 con 0x555945663c50 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555945666b40 con 0x55594565eae0 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b68c2b740 10 client.0 _ll_register_callbacks cb 0x5559455b8480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b5ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x555945668260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b5ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x555945668260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45462/0 (socket says 172.21.15.106:45462) 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555945663c50 0x55594566f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b5ea00640 1 -- 172.21.15.106:0/2062487900 learned_addr learned my addr 172.21.15.106:0/2062487900 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.699+0000 7f1b56000640 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565eae0 0x5559456687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b5f400640 1 -- 172.21.15.106:0/2062487900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565eae0 msgr2=0x5559456687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b5f400640 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565eae0 0x5559456687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b5f400640 1 -- 172.21.15.106:0/2062487900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 msgr2=0x555945668260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b5f400640 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x555945668260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b5f400640 1 -- 172.21.15.106:0/2062487900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55594539e6b0 con 0x555945663c50 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b56000640 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565eae0 0x5559456687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b5ea00640 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x555945668260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:38.703 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b5f400640 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555945663c50 0x55594566f7e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800f610 tx=0x7f1b58004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3a14f7f787520b server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b4cc00640 1 -- 172.21.15.106:0/2062487900 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58003970 con 0x555945663c50 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b68c2b740 1 -- 172.21.15.106:0/2062487900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b480fe030 con 0x555945663c50 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b4cc00640 1 -- 172.21.15.106:0/2062487900 <== mon.2 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f1b58017030 con 0x555945663c50 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b4cc00640 1 -- 172.21.15.106:0/2062487900 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b58032050 con 0x555945663c50 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b68c2b740 10 client.25436 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.700+0000 7f1b68c2b740 1 -- 172.21.15.106:0/2062487900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555945672880 con 0x555945663c50 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.701+0000 7f1b4cc00640 1 -- 172.21.15.106:0/2062487900 <== mon.2 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b480fe030 con 0x555945663c50 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.701+0000 7f1b4cc00640 1 -- 172.21.15.106:0/2062487900 <== mon.2 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1b580040c0 con 0x555945663c50 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.701+0000 7f1b68c2b740 10 client.25436 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.701+0000 7f1b68c2b740 10 client.25436 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.701+0000 7f1b68c2b740 20 client.25436 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.701+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x555945674ea0 0x555945676310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.701+0000 7f1b68c2b740 4 client.25436 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-28T12:51:38.704 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.701+0000 7f1b68c2b740 1 -- 172.21.15.106:0/2062487900 --> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559455d0210 con 0x555945674ea0 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.702+0000 7f1b5ea00640 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x555945674ea0 0x555945676310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.702+0000 7f1b5ea00640 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x555945674ea0 0x555945676310 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c608b4ecb31cad7c server_cookie=31740a04acabe8d3 in_seq=0 out_seq=0 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.702+0000 7f1b4cc00640 10 client.25436 ms_handle_connect on v2:172.21.15.129:6838/4208792665 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b4cc00640 1 -- 172.21.15.106:0/2062487900 <== mds.0 v2:172.21.15.129:6838/4208792665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5559455d0210 con 0x555945674ea0 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b4cc00640 10 client.25436 handle_command_reply: tid=0 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b68c2b740 1 client.25436 shutdown 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b68c2b740 1 -- 172.21.15.106:0/2062487900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555945663c50 msgr2=0x55594566f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555945663c50 0x55594566f7e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800f610 tx=0x7f1b58004ab0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b68c2b740 1 -- 172.21.15.106:0/2062487900 shutdown_connections 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x555945674ea0 0x555945676310 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555945663c50 0x55594566f7e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b4cc00640 0 client.25436 ms_handle_reset on v2:172.21.15.129:6838/4208792665 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565eae0 0x5559456687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.703+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/2062487900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565e0f0 0x555945668260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.705 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.704+0000 7f1b68c2b740 1 -- 172.21.15.106:0/2062487900 >> 172.21.15.106:0/2062487900 conn(0x5559455f2d10 msgr2=0x555945658eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.704+0000 7f1b68c2b740 1 -- 172.21.15.106:0/2062487900 shutdown_connections 2024-04-28T12:51:38.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.704+0000 7f1b68c2b740 1 -- 172.21.15.106:0/2062487900 wait complete. 2024-04-28T12:51:38.706 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.704+0000 7f1b68c2b740 20 client.25436 trim_cache size 0 max 16384 2024-04-28T12:51:38.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.723+0000 7f1b68c2b740 1 Processor -- start 2024-04-28T12:51:38.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.723+0000 7f1b68c2b740 1 -- start start 2024-04-28T12:51:38.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.723+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565e0f0 0x555945668580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.723+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 0x55594566f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.723+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555945663c50 0x555945658620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.723+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555945674650 con 0x555945663c50 2024-04-28T12:51:38.720 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.723+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555945674d20 con 0x55594565eae0 2024-04-28T12:51:38.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.723+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5559456753f0 con 0x55594565e0f0 2024-04-28T12:51:38.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555945663c50 0x555945658620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b56000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 0x55594566f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555945663c50 0x555945658620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45464/0 (socket says 172.21.15.106:45464) 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b5f400640 1 -- 172.21.15.106:0/3536485211 learned_addr learned my addr 172.21.15.106:0/3536485211 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b5ea00640 1 --2- 172.21.15.106:0/3536485211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565e0f0 0x555945668580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b56000640 1 -- 172.21.15.106:0/3536485211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565e0f0 msgr2=0x555945668580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b56000640 1 --2- 172.21.15.106:0/3536485211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565e0f0 0x555945668580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b56000640 1 -- 172.21.15.106:0/3536485211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555945663c50 msgr2=0x555945658620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b56000640 1 --2- 172.21.15.106:0/3536485211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555945663c50 0x555945658620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b56000640 1 -- 172.21.15.106:0/3536485211 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55594565eec0 con 0x55594565eae0 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b5f400640 1 --2- 172.21.15.106:0/3536485211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555945663c50 0x555945658620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.724+0000 7f1b56000640 1 --2- 172.21.15.106:0/3536485211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 0x55594566f7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b480bcdc0 tx=0x7f1b480eeaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468d79d8ca40510c server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b4f400640 1 -- 172.21.15.106:0/3536485211 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48049070 con 0x55594565eae0 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b4f400640 1 -- 172.21.15.106:0/3536485211 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b480fe070 con 0x55594565eae0 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b4f400640 1 -- 172.21.15.106:0/3536485211 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480bc970 con 0x55594565eae0 2024-04-28T12:51:38.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b68c2b740 1 -- 172.21.15.106:0/3536485211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 msgr2=0x55594566f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/3536485211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 0x55594566f7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1b480bcdc0 tx=0x7f1b480eeaf0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b68c2b740 1 -- 172.21.15.106:0/3536485211 shutdown_connections 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/3536485211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555945663c50 0x555945658620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/3536485211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565eae0 0x55594566f7a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/3536485211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55594565e0f0 0x555945668580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b68c2b740 1 -- 172.21.15.106:0/3536485211 >> 172.21.15.106:0/3536485211 conn(0x5559455f2d10 msgr2=0x555945667e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b68c2b740 1 -- 172.21.15.106:0/3536485211 shutdown_connections 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.725+0000 7f1b68c2b740 1 -- 172.21.15.106:0/3536485211 wait complete. 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.726+0000 7f1b68c2b740 1 -- start start 2024-04-28T12:51:38.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.726+0000 7f1b68c2b740 1 Processor -- start 2024-04-28T12:51:38.727 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.726+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565e0f0 0x55594566b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.726+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565eae0 0x55594566b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.726+0000 7f1b68c2b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 0x55594566e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.726+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555945676190 con 0x55594565eae0 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.726+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555945675ac0 con 0x55594565e0f0 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.726+0000 7f1b68c2b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555945676860 con 0x555945663c50 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.726+0000 7f1b68c2b740 10 client.0 _ll_register_callbacks cb 0x7f1b58009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b56000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565eae0 0x55594566b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b56000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565eae0 0x55594566b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45476/0 (socket says 172.21.15.106:45476) 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b56000640 1 -- 172.21.15.106:0/442252591 learned_addr learned my addr 172.21.15.106:0/442252591 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b5f400640 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 0x55594566e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b5ea00640 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565e0f0 0x55594566b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b56000640 1 -- 172.21.15.106:0/442252591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 msgr2=0x55594566e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b56000640 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 0x55594566e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b56000640 1 -- 172.21.15.106:0/442252591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565e0f0 msgr2=0x55594566b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b56000640 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565e0f0 0x55594566b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b56000640 1 -- 172.21.15.106:0/442252591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55594566ec60 con 0x55594565eae0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b56000640 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565eae0 0x55594566b6a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1b480bcb30 tx=0x7f1b48005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110161c79f6fad43 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b4cc00640 1 -- 172.21.15.106:0/442252591 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b480d6060 con 0x55594565eae0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b4cc00640 1 -- 172.21.15.106:0/442252591 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f1b480f4350 con 0x55594565eae0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.727+0000 7f1b68c2b740 1 -- 172.21.15.106:0/442252591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b480fe070 con 0x55594565eae0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b4cc00640 1 -- 172.21.15.106:0/442252591 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b48049040 con 0x55594565eae0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b68c2b740 10 client.15339 fetch_fsmap learned FSMap version 26 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b68c2b740 1 -- 172.21.15.106:0/442252591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5559456db0d0 con 0x55594565eae0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b4cc00640 1 -- 172.21.15.106:0/442252591 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b480fe070 con 0x55594565eae0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b4cc00640 1 -- 172.21.15.106:0/442252591 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1b480b94a0 con 0x55594565eae0 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b68c2b740 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-04-28T12:51:38.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b68c2b740 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b68c2b740 20 client.15339 populate_metadata read hostname 'smithi106' 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5559456df950 0x5559456dc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b68c2b740 4 client.15339 mds_command: new command op to 24578 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.728+0000 7f1b68c2b740 1 -- 172.21.15.106:0/442252591 --> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] -- command(tid 0: {"prefix": "status"}) -- 0x5559455ca080 con 0x5559456df950 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b5ea00640 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5559456df950 0x5559456dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b5ea00640 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5559456df950 0x5559456dc460 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ce1e38e90e499b server_cookie=4164aed9ec95210b in_seq=0 out_seq=0 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b4cc00640 10 client.15339 ms_handle_connect on v2:172.21.15.129:6838/4208792665 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b4cc00640 1 -- 172.21.15.106:0/442252591 <== mds.0 v2:172.21.15.129:6838/4208792665 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5559455ca080 con 0x5559456df950 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b4cc00640 10 client.15339 handle_command_reply: tid=0 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b68c2b740 1 client.15339 shutdown 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b68c2b740 1 -- 172.21.15.106:0/442252591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565eae0 msgr2=0x55594566b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565eae0 0x55594566b6a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1b480bcb30 tx=0x7f1b48005de0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b68c2b740 1 -- 172.21.15.106:0/442252591 shutdown_connections 2024-04-28T12:51:38.730 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.129:6838/4208792665,v1:172.21.15.129:6839/4208792665] conn(0x5559456df950 0x5559456dc460 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55594565eae0 0x55594566b6a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b4cc00640 0 client.15339 ms_handle_reset on v2:172.21.15.129:6838/4208792665 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.729+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555945663c50 0x55594566e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b68c2b740 1 --2- 172.21.15.106:0/442252591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55594565e0f0 0x55594566b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b68c2b740 1 -- 172.21.15.106:0/442252591 >> 172.21.15.106:0/442252591 conn(0x5559455f2d10 msgr2=0x55594565cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b68c2b740 1 -- 172.21.15.106:0/442252591 shutdown_connections 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b68c2b740 1 -- 172.21.15.106:0/442252591 wait complete. 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b68c2b740 20 client.15339 trim_cache size 0 max 16384 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "e5e35dfa-055b-11ef-bc93-c7b262605968", 2024-04-28T12:51:38.731 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 24578, 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 163.62548154999999, 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 26, 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 93, 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 169.98528134099999 2024-04-28T12:51:38.732 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 -- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1b440a3c40 msgr2=0x7f1b440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1b440a3c40 0x7f1b440a6100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55594565f340 tx=0x7f1b58009500 comp rx=0 tx=0).stop 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 -- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 msgr2=0x7f1b6019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 0x7f1b6019b350 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1b48005ab0 tx=0x7f1b48003b90 comp rx=0 tx=0).stop 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 -- 172.21.15.106:0/3423016203 shutdown_connections 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f1b440a3c40 0x7f1b440a6100 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b60101950 0x7f1b6019b350 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b601022f0 0x7f1b601a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 --2- 172.21.15.106:0/3423016203 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b601008e0 0x7f1b6019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.730+0000 7f1b65e00640 1 -- 172.21.15.106:0/3423016203 >> 172.21.15.106:0/3423016203 conn(0x7f1b600fc120 msgr2=0x7f1b600fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.731+0000 7f1b65e00640 1 -- 172.21.15.106:0/3423016203 shutdown_connections 2024-04-28T12:51:38.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:38.731+0000 7f1b65e00640 1 -- 172.21.15.106:0/3423016203 wait complete. 2024-04-28T12:51:38.744 INFO:teuthology.run_tasks:Running task sequential... 2024-04-28T12:51:38.754 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-28T12:51:38.754 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-28T12:51:38.756 INFO:teuthology.task.print:Enabling mgr modules 2024-04-28T12:51:38.757 INFO:teuthology.run_tasks:Running task kclient... 2024-04-28T12:51:38.793 INFO:tasks.kclient:Mounting kernel clients... 2024-04-28T12:51:38.793 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-28T12:51:38.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:38.793 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-28T12:51:38.845 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-28T12:51:38.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:38.847 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-28T12:51:38.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:38.929 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-28T12:51:39.051 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-28T12:51:39.052 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-28T12:51:39.052 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T12:51:39.052 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387 shell --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-28T12:51:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/mon.a/config 2024-04-28T12:51:39.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:39 smithi171 ceph-mon[63127]: pgmap v404: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:51:39.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:39 smithi129 ceph-mon[62418]: pgmap v404: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:51:39.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:39 smithi106 ceph-mon[28081]: pgmap v404: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:51:41.494 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:41 smithi106 ceph-mon[28081]: pgmap v405: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:51:41.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:41 smithi171 ceph-mon[63127]: pgmap v405: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:51:41.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:41 smithi129 ceph-mon[62418]: pgmap v405: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:51:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.808+0000 7fcefbe00700 1 -- 172.21.15.106:0/2763738634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcef4109a50 msgr2=0x7fcef4109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.808+0000 7fcefbe00700 1 --2- 172.21.15.106:0/2763738634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcef4109a50 0x7fcef4109ed0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcee0009990 tx=0x7fcee0009ca0 comp rx=0 tx=0).stop 2024-04-28T12:51:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.808+0000 7fcefbe00700 1 -- 172.21.15.106:0/2763738634 shutdown_connections 2024-04-28T12:51:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.808+0000 7fcefbe00700 1 --2- 172.21.15.106:0/2763738634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcef410a410 0x7fcef4116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.808+0000 7fcefbe00700 1 --2- 172.21.15.106:0/2763738634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcef410fa50 0x7fcef410fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.808+0000 7fcefbe00700 1 --2- 172.21.15.106:0/2763738634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcef4109a50 0x7fcef4109ed0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.808+0000 7fcefbe00700 1 -- 172.21.15.106:0/2763738634 >> 172.21.15.106:0/2763738634 conn(0x7fcef4076bf0 msgr2=0x7fcef4108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:41.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.808+0000 7fcefbe00700 1 -- 172.21.15.106:0/2763738634 shutdown_connections 2024-04-28T12:51:41.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.808+0000 7fcefbe00700 1 -- 172.21.15.106:0/2763738634 wait complete. 2024-04-28T12:51:41.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.809+0000 7fcefbe00700 1 Processor -- start 2024-04-28T12:51:41.807 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.809+0000 7fcefbe00700 1 -- start start 2024-04-28T12:51:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcefbe00700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcef4109a50 0x7fcef41a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcefbe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcef410a410 0x7fcef41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcefbe00700 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcef410fa50 0x7fcef41a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcefbe00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4119c90 con 0x7fcef410a410 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcefbe00700 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4119b10 con 0x7fcef410fa50 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcefbe00700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcef4119e10 con 0x7fcef4109a50 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcef8a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcef410a410 0x7fcef41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcef9400700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcef4109a50 0x7fcef41a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcef8a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcef410a410 0x7fcef41a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37514/0 (socket says 172.21.15.106:37514) 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcef8a00700 1 -- 172.21.15.106:0/3639496166 learned_addr learned my addr 172.21.15.106:0/3639496166 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.810+0000 7fcef9e00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcef410fa50 0x7fcef41a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef9400700 1 -- 172.21.15.106:0/3639496166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcef410fa50 msgr2=0x7fcef41a4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef9400700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcef410fa50 0x7fcef41a4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef9400700 1 -- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcef410a410 msgr2=0x7fcef41a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef9400700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcef410a410 0x7fcef41a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef9400700 1 -- 172.21.15.106:0/3639496166 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef41a50a0 con 0x7fcef4109a50 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef8a00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcef410a410 0x7fcef41a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef9e00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcef410fa50 0x7fcef41a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:41.809 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef9400700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcef4109a50 0x7fcef41a00e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e9f0 tx=0x7fcee800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:41.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef2000700 1 -- 172.21.15.106:0/3639496166 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee800c9c0 con 0x7fcef4109a50 2024-04-28T12:51:41.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef2000700 1 -- 172.21.15.106:0/3639496166 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee800cb60 con 0x7fcef4109a50 2024-04-28T12:51:41.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcef2000700 1 -- 172.21.15.106:0/3639496166 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8018680 con 0x7fcef4109a50 2024-04-28T12:51:41.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef41a5390 con 0x7fcef4109a50 2024-04-28T12:51:41.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.811+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef41acc10 con 0x7fcef4109a50 2024-04-28T12:51:41.810 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.812+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef406a110 con 0x7fcef4109a50 2024-04-28T12:51:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.814+0000 7fcef2000700 1 -- 172.21.15.106:0/3639496166 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fcee8010430 con 0x7fcef4109a50 2024-04-28T12:51:41.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.816+0000 7fcef2000700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcedc0a3cd0 0x7fcedc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:41.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.816+0000 7fcef2000700 1 -- 172.21.15.106:0/3639496166 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fcee80d0db0 con 0x7fcef4109a50 2024-04-28T12:51:41.813 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.816+0000 7fcef8a00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcedc0a3cd0 0x7fcedc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:41.814 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.817+0000 7fcef8a00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcedc0a3cd0 0x7fcedc0a6190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcef4078ac0 tx=0x7fcee0009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:41.815 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:41.818+0000 7fcef2000700 1 -- 172.21.15.106:0/3639496166 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fcee809d220 con 0x7fcef4109a50 2024-04-28T12:51:42.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.096+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcef41a13f0 con 0x7fcef4109a50 2024-04-28T12:51:42.093 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.097+0000 7fcef2000700 1 -- 172.21.15.106:0/3639496166 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7fcee809cb20 con 0x7fcef4109a50 2024-04-28T12:51:42.094 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.099+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcedc0a3cd0 msgr2=0x7fcedc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.099+0000 7fcefbe00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcedc0a3cd0 0x7fcedc0a6190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcef4078ac0 tx=0x7fcee0009510 comp rx=0 tx=0).stop 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.099+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcef4109a50 msgr2=0x7fcef41a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.099+0000 7fcefbe00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcef4109a50 0x7fcef41a00e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e9f0 tx=0x7fcee800ed00 comp rx=0 tx=0).stop 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.100+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 shutdown_connections 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.100+0000 7fcefbe00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcef410fa50 0x7fcef41a4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.100+0000 7fcefbe00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fcedc0a3cd0 0x7fcedc0a6190 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.100+0000 7fcefbe00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcef4109a50 0x7fcef41a00e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.100+0000 7fcefbe00700 1 --2- 172.21.15.106:0/3639496166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcef410a410 0x7fcef41a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:42.097 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.100+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 >> 172.21.15.106:0/3639496166 conn(0x7fcef4076bf0 msgr2=0x7fcef4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:42.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.100+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 shutdown_connections 2024-04-28T12:51:42.098 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:42.100+0000 7fcefbe00700 1 -- 172.21.15.106:0/3639496166 wait complete. 2024-04-28T12:51:42.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:42 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3639496166' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-28T12:51:42.522 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:42 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3639496166' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-28T12:51:42.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:42 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3639496166' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-28T12:51:42.922 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-28T12:51:42.922 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-28T12:51:42.922 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2024-04-28T12:51:42.923 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-28T12:51:42.923 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-28T12:51:42.923 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-28T12:51:42.923 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-28T12:51:42.923 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-28T12:51:42.923 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-28T12:51:42.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:42.923 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2024-04-28T12:51:42.978 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-28T12:51:42.979 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2024-04-28T12:51:42.980 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-28T12:51:42.980 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2024-04-28T12:51:42.980 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-28T12:51:42.980 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 41674sec preferred_lft 41674sec 2024-04-28T12:51:42.980 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2024-04-28T12:51:42.980 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2024-04-28T12:51:42.981 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-28T12:51:42.981 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:51:42.981 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:51:42.981 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add name ceph-brx type bridge 2024-04-28T12:51:42.981 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr flush dev ceph-brx 2024-04-28T12:51:42.981 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set ceph-brx up 2024-04-28T12:51:42.981 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-28T12:51:42.981 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:51:43.083 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:51:43.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:51:43.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:43.223 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-28T12:51:43.289 INFO:teuthology.orchestra.run.smithi106.stdout:1 2024-04-28T12:51:43.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:43.293 DEBUG:teuthology.orchestra.run.smithi106:> route 2024-04-28T12:51:43.342 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2024-04-28T12:51:43.342 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-28T12:51:43.342 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-28T12:51:43.342 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-28T12:51:43.342 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-28T12:51:43.343 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:51:43.343 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:51:43.343 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-28T12:51:43.343 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-28T12:51:43.343 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-28T12:51:43.343 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:51:43.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:51:43.467 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:43 smithi106 ceph-mon[28081]: pgmap v406: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:51:43.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:43 smithi129 ceph-mon[62418]: pgmap v406: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:51:43.538 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:51:43.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:43.548 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-28T12:51:43.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:43.598 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2024-04-28T12:51:43.649 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:51:43.649 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:51:43.649 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T12:51:43.649 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-28T12:51:43.649 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:51:43.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:43 smithi171 ceph-mon[63127]: pgmap v406: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:51:43.752 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:51:43.820 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:51:43.830 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-28T12:51:43.830 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:51:43.830 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:51:43.830 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-28T12:51:43.831 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-28T12:51:43.831 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-28T12:51:43.831 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-28T12:51:43.831 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-28T12:51:43.831 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:51:43.936 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:51:44.088 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:51:44.097 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-28T12:51:44.097 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2024-04-28T12:51:44.097 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2024-04-28T12:51:44.097 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.0 master ceph-brx 2024-04-28T12:51:44.097 DEBUG:teuthology.orchestra.run.smithi106:> ') 2024-04-28T12:51:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:51:44.284 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:44 smithi106 ceph-mon[28081]: pgmap v407: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:51:44.287 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:51:44.296 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:51:44.296 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-28T12:51:44.340 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-28T12:51:44.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-28T12:51:44.340 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-28T12:51:44.436 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-28T12:51:44.436 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-28T12:51:44.437 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/be9407c1-65bc-4ae7-81c7-e36ca57f5e10 2024-04-28T12:51:44.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:44.437 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/be9407c1-65bc-4ae7-81c7-e36ca57f5e10 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-28T12:51:44.566 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-28T12:51:44.566 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-28T12:51:44.567 INFO:teuthology.orchestra.run.smithi106.stdout:invalid new device string format 2024-04-28T12:51:44.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:44 smithi171 ceph-mon[63127]: pgmap v407: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:51:44.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:44 smithi129 ceph-mon[62418]: pgmap v407: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:51:45.284 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-28T12:51:45.284 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-28T12:51:45.284 INFO:teuthology.orchestra.run.smithi106.stdout:invalid new device string format 2024-04-28T12:51:45.285 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.129:3300,172.21.15.171:3300" 2024-04-28T12:51:45.285 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.106:3300,172.21.15.129:3300,172.21.15.171:3300:/volumes/qa/sv_1/be9407c1-65bc-4ae7-81c7-e36ca57f5e10 2024-04-28T12:51:45.285 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=e5e35dfa-055b-11ef-bc93-c7b262605968" will pass to kernel 2024-04-28T12:51:45.288 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-28T12:51:45.288 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2024-04-28T12:51:45.288 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-28T12:51:45.288 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2024-04-28T12:51:45.288 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-28T12:51:45.292 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-28T12:51:45.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T12:51:45.292 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-28T12:51:45.384 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T12:51:45.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T12:51:45.497 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:51:45.507 INFO:teuthology.orchestra.run.smithi106.stdout:client25444 2024-04-28T12:51:45.507 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T12:51:45.507 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T12:51:45.557 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T12:51:45.710 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T12:51:45.715 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:51:45.724 INFO:teuthology.orchestra.run.smithi106.stdout:client25444 2024-04-28T12:51:45.725 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-28T12:51:45.829 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-28T12:51:45.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:51:45.844 INFO:teuthology.orchestra.run.smithi106.stdout:client25444 2024-04-28T12:51:45.844 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968.client25444/status') 2024-04-28T12:51:45.944 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-28T12:51:45.975 INFO:teuthology.orchestra.run.smithi106.stdout:instance: client.25444 (3)172.21.15.106:0/481351348 2024-04-28T12:51:45.975 INFO:teuthology.orchestra.run.smithi106.stdout:blocklisted: false 2024-04-28T12:51:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2024-04-28T12:51:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2024-04-28T12:51:45.975 INFO:teuthology.orchestra.run.smithi106.stderr:71 bytes copied, 8.0758e-05 s, 879 kB/s 2024-04-28T12:51:45.978 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-28T12:51:45.989 INFO:teuthology.run_tasks:Running task exec... 2024-04-28T12:51:46.008 INFO:teuthology.task.exec:Executing custom commands... 2024-04-28T12:51:46.008 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi106.front.sepia.ceph.com 2024-04-28T12:51:46.008 DEBUG:teuthology.orchestra.run.smithi106:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-28T12:51:46.210 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.213+0000 7f9eeb200640 1 -- 172.21.15.106:0/4040417750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1043a0 msgr2=0x7f9eec10ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:46.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.213+0000 7f9eeb200640 1 --2- 172.21.15.106:0/4040417750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1043a0 0x7f9eec10ac30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009900 tx=0x7f9ee402a410 comp rx=0 tx=0).stop 2024-04-28T12:51:46.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.213+0000 7f9eeb200640 1 -- 172.21.15.106:0/4040417750 shutdown_connections 2024-04-28T12:51:46.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.213+0000 7f9eeb200640 1 --2- 172.21.15.106:0/4040417750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1043a0 0x7f9eec10ac30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:46.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.213+0000 7f9eeb200640 1 --2- 172.21.15.106:0/4040417750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9eec1039e0 0x7f9eec103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:46.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.213+0000 7f9eeb200640 1 --2- 172.21.15.106:0/4040417750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9eec1027f0 0x7f9eec102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:46.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.213+0000 7f9eeb200640 1 -- 172.21.15.106:0/4040417750 >> 172.21.15.106:0/4040417750 conn(0x7f9eec0fdd60 msgr2=0x7f9eec1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:46.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.213+0000 7f9eeb200640 1 -- 172.21.15.106:0/4040417750 shutdown_connections 2024-04-28T12:51:46.211 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.213+0000 7f9eeb200640 1 -- 172.21.15.106:0/4040417750 wait complete. 2024-04-28T12:51:46.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.214+0000 7f9eeb200640 1 Processor -- start 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.214+0000 7f9eeb200640 1 -- start start 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9eeb200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1027f0 0x7f9eec19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9eeb200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9eec1039e0 0x7f9eec19b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9eeb200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9eec1043a0 0x7f9eec1a2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9eeb200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10e370 con 0x7f9eec1027f0 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9eeb200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10e1f0 con 0x7f9eec1039e0 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9eeb200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec10e4f0 con 0x7f9eec1043a0 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1027f0 0x7f9eec19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1027f0 0x7f9eec19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37524/0 (socket says 172.21.15.106:37524) 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9e00640 1 -- 172.21.15.106:0/2604482575 learned_addr learned my addr 172.21.15.106:0/2604482575 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9eec1039e0 0x7f9eec19b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:46.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9eea800640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9eec1043a0 0x7f9eec1a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9e00640 1 -- 172.21.15.106:0/2604482575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9eec1043a0 msgr2=0x7f9eec1a2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9e00640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9eec1043a0 0x7f9eec1a2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9e00640 1 -- 172.21.15.106:0/2604482575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9eec1039e0 msgr2=0x7f9eec19b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9e00640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9eec1039e0 0x7f9eec19b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9e00640 1 -- 172.21.15.106:0/2604482575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec1a27c0 con 0x7f9eec1027f0 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.215+0000 7f9ee9400640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9eec1039e0 0x7f9eec19b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.216+0000 7f9ee9e00640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1027f0 0x7f9eec19aba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00ebc0 tx=0x7f9edc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.216+0000 7f9ee2a00640 1 -- 172.21.15.106:0/2604482575 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc003b70 con 0x7f9eec1027f0 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.216+0000 7f9ee2a00640 1 -- 172.21.15.106:0/2604482575 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9edc003d10 con 0x7f9eec1027f0 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.216+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec1a2ab0 con 0x7f9eec1027f0 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.216+0000 7f9ee2a00640 1 -- 172.21.15.106:0/2604482575 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc019800 con 0x7f9eec1027f0 2024-04-28T12:51:46.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.216+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec1a3000 con 0x7f9eec1027f0 2024-04-28T12:51:46.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.217+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec1a31d0 con 0x7f9eec1027f0 2024-04-28T12:51:46.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.219+0000 7f9ee2a00640 1 -- 172.21.15.106:0/2604482575 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9edc0199a0 con 0x7f9eec1027f0 2024-04-28T12:51:46.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.221+0000 7f9ee2a00640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ed00a3c30 0x7f9ed00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:46.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.221+0000 7f9ee2a00640 1 -- 172.21.15.106:0/2604482575 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f9edc0d1220 con 0x7f9eec1027f0 2024-04-28T12:51:46.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.221+0000 7f9ee9400640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ed00a3c30 0x7f9ed00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:46.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.222+0000 7f9ee9400640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ed00a3c30 0x7f9ed00a60f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9eec19c0b0 tx=0x7f9ed4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:46.222 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.225+0000 7f9ee2a00640 1 -- 172.21.15.106:0/2604482575 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f9edc09d690 con 0x7f9eec1027f0 2024-04-28T12:51:46.471 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:46.474+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f9eec1a33a0 con 0x7f9eec1027f0 2024-04-28T12:51:47.184 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.186+0000 7f9ee2a00640 1 -- 172.21.15.106:0/2604482575 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v27) ==== 118+0+0 (secure 0 0 0) 0x7f9edc09cf90 con 0x7f9eec1027f0 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.191+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ed00a3c30 msgr2=0x7f9ed00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.191+0000 7f9eeb200640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ed00a3c30 0x7f9ed00a60f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9eec19c0b0 tx=0x7f9ed4005f90 comp rx=0 tx=0).stop 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.191+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1027f0 msgr2=0x7f9eec19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.191+0000 7f9eeb200640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1027f0 0x7f9eec19aba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00ebc0 tx=0x7f9edc00c6a0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.192+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 shutdown_connections 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.192+0000 7f9eeb200640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f9ed00a3c30 0x7f9ed00a60f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.192+0000 7f9eeb200640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9eec1043a0 0x7f9eec1a2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.192+0000 7f9eeb200640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9eec1039e0 0x7f9eec19b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.192+0000 7f9eeb200640 1 --2- 172.21.15.106:0/2604482575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9eec1027f0 0x7f9eec19aba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.189 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.192+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 >> 172.21.15.106:0/2604482575 conn(0x7f9eec0fdd60 msgr2=0x7f9eec074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:47.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.192+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 shutdown_connections 2024-04-28T12:51:47.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.192+0000 7f9eeb200640 1 -- 172.21.15.106:0/2604482575 wait complete. 2024-04-28T12:51:47.214 DEBUG:teuthology.orchestra.run.smithi106:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-28T12:51:47.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.416+0000 7fe05f800640 1 -- 172.21.15.106:0/2638334811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05810a1b0 msgr2=0x7fe05810a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:47.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.416+0000 7fe05f800640 1 --2- 172.21.15.106:0/2638334811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05810a1b0 0x7fe05810a630 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe040009900 tx=0x7fe04002a410 comp rx=0 tx=0).stop 2024-04-28T12:51:47.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.416+0000 7fe05f800640 1 -- 172.21.15.106:0/2638334811 shutdown_connections 2024-04-28T12:51:47.413 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.416+0000 7fe05f800640 1 --2- 172.21.15.106:0/2638334811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05810a1b0 0x7fe05810a630 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.416+0000 7fe05f800640 1 --2- 172.21.15.106:0/2638334811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05810ab70 0x7fe058111400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.416+0000 7fe05f800640 1 --2- 172.21.15.106:0/2638334811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe058108fc0 0x7fe0581093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.416+0000 7fe05f800640 1 -- 172.21.15.106:0/2638334811 >> 172.21.15.106:0/2638334811 conn(0x7fe058077f80 msgr2=0x7fe05807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T12:51:47.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.416+0000 7fe05f800640 1 -- 172.21.15.106:0/2638334811 shutdown_connections 2024-04-28T12:51:47.414 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.416+0000 7fe05f800640 1 -- 172.21.15.106:0/2638334811 wait complete. 2024-04-28T12:51:47.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.417+0000 7fe05f800640 1 Processor -- start 2024-04-28T12:51:47.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.417+0000 7fe05f800640 1 -- start start 2024-04-28T12:51:47.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.417+0000 7fe05f800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe058108fc0 0x7fe05819f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:47.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.417+0000 7fe05f800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05810a1b0 0x7fe05819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:47.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.417+0000 7fe05f800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05810ab70 0x7fe0581a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:47.416 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.417+0000 7fe05f800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe058114b00 con 0x7fe05810a1b0 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.417+0000 7fe05f800640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe058114980 con 0x7fe05810ab70 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.417+0000 7fe05f800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe058114c80 con 0x7fe058108fc0 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe04fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05810a1b0 0x7fe05819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe05ce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe058108fc0 0x7fe05819f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe04fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05810a1b0 0x7fe05819f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:37538/0 (socket says 172.21.15.106:37538) 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe05ce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe058108fc0 0x7fe05819f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:44276/0 (socket says 172.21.15.106:44276) 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe04fe00640 1 -- 172.21.15.106:0/3510684457 learned_addr learned my addr 172.21.15.106:0/3510684457 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe05d800640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05810ab70 0x7fe0581a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe05ce00640 1 -- 172.21.15.106:0/3510684457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05810ab70 msgr2=0x7fe0581a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe05ce00640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe05810ab70 0x7fe0581a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe05ce00640 1 -- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05810a1b0 msgr2=0x7fe05819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe05ce00640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe05810a1b0 0x7fe05819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe05ce00640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0581a6d50 con 0x7fe058108fc0 2024-04-28T12:51:47.417 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.418+0000 7fe05ce00640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe058108fc0 0x7fe05819f130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe04800e9c0 tx=0x7fe04800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:47.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.419+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04800ccc0 con 0x7fe058108fc0 2024-04-28T12:51:47.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.419+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0480035f0 con 0x7fe058108fc0 2024-04-28T12:51:47.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.419+0000 7fe05f800640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0581a6fe0 con 0x7fe058108fc0 2024-04-28T12:51:47.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.419+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04800ce60 con 0x7fe058108fc0 2024-04-28T12:51:47.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.419+0000 7fe05f800640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0581a74a0 con 0x7fe058108fc0 2024-04-28T12:51:47.418 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.420+0000 7fe05f800640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0581102e0 con 0x7fe058108fc0 2024-04-28T12:51:47.419 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.422+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fe048004000 con 0x7fe058108fc0 2024-04-28T12:51:47.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.423+0000 7fe056000640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0a3d70 0x7fe03c0a6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T12:51:47.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.424+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fe0480d0c60 con 0x7fe058108fc0 2024-04-28T12:51:47.420 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.424+0000 7fe04fe00640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0a3d70 0x7fe03c0a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T12:51:47.421 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.424+0000 7fe04fe00640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0a3d70 0x7fe03c0a6230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe040009230 tx=0x7fe0400058d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T12:51:47.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.428+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fe04809d0d0 con 0x7fe058108fc0 2024-04-28T12:51:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:47 smithi171 ceph-mon[63127]: pgmap v408: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:51:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:47 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2604482575' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-28T12:51:47.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:47 smithi129 ceph-mon[62418]: pgmap v408: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:51:47.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:47 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2604482575' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-28T12:51:47.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:47 smithi106 ceph-mon[28081]: pgmap v408: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:51:47.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:47 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2604482575' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-28T12:51:47.688 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:51:47.691+0000 7fe05f800640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fe058062e10 con 0x7fe03c0a3d70 2024-04-28T12:51:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:48 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2604482575' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-28T12:51:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:48 smithi171 ceph-mon[63127]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-28T12:51:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:51:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:48 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:48 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2604482575' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-28T12:51:48.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:48 smithi129 ceph-mon[62418]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-28T12:51:48.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:51:48.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:48 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:48 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2604482575' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-28T12:51:48.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:48 smithi106 ceph-mon[28081]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-28T12:51:48.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:51:48.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:48.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:48 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:49.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:49 smithi129 ceph-mon[62418]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:49.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:49 smithi129 ceph-mon[62418]: pgmap v409: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-28T12:51:49.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:49.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:49 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:49.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:49 smithi171 ceph-mon[63127]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:49.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:49 smithi171 ceph-mon[63127]: pgmap v409: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-28T12:51:49.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:49.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:49 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:49.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:49 smithi106 ceph-mon[28081]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T12:51:49.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:49 smithi106 ceph-mon[28081]: pgmap v409: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-28T12:51:49.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:49.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:49 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:50 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:50 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:50 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:50 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:50 smithi171 ceph-mon[63127]: pgmap v410: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-28T12:51:51.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:50 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:50 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:50 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.264 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:50 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:50 smithi129 ceph-mon[62418]: pgmap v410: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-28T12:51:51.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:50 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:50 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:50 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.325 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:50 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:51.326 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:50 smithi106 ceph-mon[28081]: pgmap v410: 129 pgs: 129 active+clean; 635 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-28T12:51:52.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:52.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:52 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:52.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:52.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:52 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:52.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:52.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:52 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:53.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:53 smithi171 ceph-mon[63127]: pgmap v411: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 852 B/s wr, 11 op/s 2024-04-28T12:51:53.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:53 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:51:53.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:53 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:51:53.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:53 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:53.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:53 smithi129 ceph-mon[62418]: pgmap v411: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 852 B/s wr, 11 op/s 2024-04-28T12:51:53.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:53 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:51:53.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:53 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:51:53.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:53 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:53.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:53 smithi106 ceph-mon[28081]: pgmap v411: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 852 B/s wr, 11 op/s 2024-04-28T12:51:53.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:53 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:51:53.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:53 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:51:53.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:53 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:51:55.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:55 smithi171 ceph-mon[63127]: pgmap v412: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 852 B/s wr, 10 op/s 2024-04-28T12:51:55.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:55 smithi129 ceph-mon[62418]: pgmap v412: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 852 B/s wr, 10 op/s 2024-04-28T12:51:55.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:55 smithi106 ceph-mon[28081]: pgmap v412: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 852 B/s wr, 10 op/s 2024-04-28T12:51:57.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:57 smithi171 ceph-mon[63127]: pgmap v413: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-28T12:51:57.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:57 smithi129 ceph-mon[62418]: pgmap v413: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-28T12:51:57.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:57 smithi106 ceph-mon[28081]: pgmap v413: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 852 B/s wr, 9 op/s 2024-04-28T12:51:59.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:51:59 smithi171 ceph-mon[63127]: pgmap v414: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 852 B/s wr, 11 op/s 2024-04-28T12:51:59.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:51:59 smithi129 ceph-mon[62418]: pgmap v414: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 852 B/s wr, 11 op/s 2024-04-28T12:51:59.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:51:59 smithi106 ceph-mon[28081]: pgmap v414: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 852 B/s wr, 11 op/s 2024-04-28T12:52:01.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:01 smithi129 ceph-mon[62418]: pgmap v415: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-28T12:52:01.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:01 smithi106 ceph-mon[28081]: pgmap v415: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-28T12:52:01.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:01 smithi171 ceph-mon[63127]: pgmap v415: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-28T12:52:02.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:02 smithi129 ceph-mon[62418]: pgmap v416: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-28T12:52:02.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:02 smithi106 ceph-mon[28081]: pgmap v416: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-28T12:52:02.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:02 smithi171 ceph-mon[63127]: pgmap v416: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-28T12:52:05.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:05 smithi171 ceph-mon[63127]: pgmap v417: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:05.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:05 smithi129 ceph-mon[62418]: pgmap v417: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:05.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:05 smithi106 ceph-mon[28081]: pgmap v417: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:07.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:07 smithi171 ceph-mon[63127]: pgmap v418: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:07.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:07 smithi129 ceph-mon[62418]: pgmap v418: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:07.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:07 smithi106 ceph-mon[28081]: pgmap v418: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:09.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:09 smithi171 ceph-mon[63127]: pgmap v419: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:09.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:09 smithi129 ceph-mon[62418]: pgmap v419: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:09.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:09 smithi106 ceph-mon[28081]: pgmap v419: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:11.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:11 smithi171 ceph-mon[63127]: pgmap v420: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:52:11.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:11 smithi129 ceph-mon[62418]: pgmap v420: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:52:11.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:11 smithi106 ceph-mon[28081]: pgmap v420: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:52:13.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:13 smithi171 ceph-mon[63127]: pgmap v421: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:52:13.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:13 smithi129 ceph-mon[62418]: pgmap v421: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:52:13.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:13 smithi106 ceph-mon[28081]: pgmap v421: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:52:15.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:15 smithi171 ceph-mon[63127]: pgmap v422: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:52:15.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:15 smithi129 ceph-mon[62418]: pgmap v422: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:52:15.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:15 smithi106 ceph-mon[28081]: pgmap v422: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:52:17.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:17 smithi171 ceph-mon[63127]: pgmap v423: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T12:52:17.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:17 smithi129 ceph-mon[62418]: pgmap v423: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T12:52:17.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:17 smithi106 ceph-mon[28081]: pgmap v423: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T12:52:19.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:19 smithi171 ceph-mon[63127]: pgmap v424: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:52:19.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:19 smithi129 ceph-mon[62418]: pgmap v424: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:52:19.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:19 smithi106 ceph-mon[28081]: pgmap v424: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-28T12:52:21.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:21 smithi171 ceph-mon[63127]: pgmap v425: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:52:21.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:21 smithi129 ceph-mon[62418]: pgmap v425: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:52:21.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:21 smithi106 ceph-mon[28081]: pgmap v425: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:52:23.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:23 smithi171 ceph-mon[63127]: pgmap v426: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:52:23.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:23 smithi129 ceph-mon[62418]: pgmap v426: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:52:23.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:23 smithi106 ceph-mon[28081]: pgmap v426: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:52:25.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:25 smithi171 ceph-mon[63127]: pgmap v427: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:25.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:25 smithi129 ceph-mon[62418]: pgmap v427: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:25.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:25 smithi106 ceph-mon[28081]: pgmap v427: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:27.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:27 smithi171 ceph-mon[63127]: pgmap v428: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:27.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:27 smithi129 ceph-mon[62418]: pgmap v428: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:27.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:27 smithi106 ceph-mon[28081]: pgmap v428: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:29.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:29 smithi129 ceph-mon[62418]: pgmap v429: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:29.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:29 smithi106 ceph-mon[28081]: pgmap v429: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:29.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:29 smithi171 ceph-mon[63127]: pgmap v429: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:31.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:31 smithi129 ceph-mon[62418]: pgmap v430: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:52:31.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:31 smithi106 ceph-mon[28081]: pgmap v430: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:52:31.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:31 smithi171 ceph-mon[63127]: pgmap v430: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:52:32.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:32 smithi106 ceph-mon[28081]: pgmap v431: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:52:32.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:32 smithi171 ceph-mon[63127]: pgmap v431: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:52:32.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:32 smithi129 ceph-mon[62418]: pgmap v431: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:52:35.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:35 smithi171 ceph-mon[63127]: pgmap v432: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:35.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:35 smithi129 ceph-mon[62418]: pgmap v432: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:35.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:35 smithi106 ceph-mon[28081]: pgmap v432: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:37.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:37 smithi171 ceph-mon[63127]: pgmap v433: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:37.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:37 smithi129 ceph-mon[62418]: pgmap v433: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:37.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:37 smithi106 ceph-mon[28081]: pgmap v433: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:39.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:39 smithi171 ceph-mon[63127]: pgmap v434: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:39.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:39 smithi129 ceph-mon[62418]: pgmap v434: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:39.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:39 smithi106 ceph-mon[28081]: pgmap v434: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:41.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:41 smithi171 ceph-mon[63127]: pgmap v435: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:52:41.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:41 smithi129 ceph-mon[62418]: pgmap v435: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:52:41.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:41 smithi106 ceph-mon[28081]: pgmap v435: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:52:43.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:43 smithi171 ceph-mon[63127]: pgmap v436: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:52:43.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:43 smithi129 ceph-mon[62418]: pgmap v436: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:52:43.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:43 smithi106 ceph-mon[28081]: pgmap v436: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:52:45.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:45 smithi171 ceph-mon[63127]: pgmap v437: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:45.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:45 smithi129 ceph-mon[62418]: pgmap v437: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:45.575 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:45 smithi106 ceph-mon[28081]: pgmap v437: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:52:47.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:47 smithi171 ceph-mon[63127]: pgmap v438: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:47.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:47 smithi129 ceph-mon[62418]: pgmap v438: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:47.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:47 smithi106 ceph-mon[28081]: pgmap v438: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:52:49.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:49 smithi171 ceph-mon[63127]: pgmap v439: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:49.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:49 smithi129 ceph-mon[62418]: pgmap v439: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:49.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:49 smithi106 ceph-mon[28081]: pgmap v439: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:52:51.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:51 smithi129 ceph-mon[62418]: pgmap v440: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-28T12:52:51.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:51 smithi106 ceph-mon[28081]: pgmap v440: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-28T12:52:51.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:51 smithi171 ceph-mon[63127]: pgmap v440: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-28T12:52:53.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:53 smithi129 ceph-mon[62418]: pgmap v441: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-28T12:52:53.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:53 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:52:53.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:53 smithi106 ceph-mon[28081]: pgmap v441: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-28T12:52:53.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:53 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:52:53.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:53 smithi171 ceph-mon[63127]: pgmap v441: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-28T12:52:53.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:53 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:52:54.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:52:54.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:52:54.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:52:54.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:52:54.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:52:54.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:52:54.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:52:54.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:52:54.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:52:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:55 smithi106 ceph-mon[28081]: pgmap v442: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-28T12:52:55.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:55 smithi171 ceph-mon[63127]: pgmap v442: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-28T12:52:55.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:55 smithi129 ceph-mon[62418]: pgmap v442: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-28T12:52:56.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:56 smithi106 ceph-mon[28081]: pgmap v443: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-28T12:52:56.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:56 smithi171 ceph-mon[63127]: pgmap v443: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-28T12:52:56.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:56 smithi129 ceph-mon[62418]: pgmap v443: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-28T12:52:59.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:52:59 smithi171 ceph-mon[63127]: pgmap v444: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-28T12:52:59.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:52:59 smithi129 ceph-mon[62418]: pgmap v444: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-28T12:52:59.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:52:59 smithi106 ceph-mon[28081]: pgmap v444: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-28T12:53:01.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:01 smithi171 ceph-mon[63127]: pgmap v445: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-28T12:53:01.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:01 smithi129 ceph-mon[62418]: pgmap v445: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-28T12:53:01.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:01 smithi106 ceph-mon[28081]: pgmap v445: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-04-28T12:53:03.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:03 smithi171 ceph-mon[63127]: pgmap v446: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:03.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:03 smithi129 ceph-mon[62418]: pgmap v446: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:03.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:03 smithi106 ceph-mon[28081]: pgmap v446: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:05.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:05 smithi171 ceph-mon[63127]: pgmap v447: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:05.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:05 smithi129 ceph-mon[62418]: pgmap v447: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:05.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:05 smithi106 ceph-mon[28081]: pgmap v447: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:07.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:07 smithi171 ceph-mon[63127]: pgmap v448: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:07.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:07 smithi129 ceph-mon[62418]: pgmap v448: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:07.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:07 smithi106 ceph-mon[28081]: pgmap v448: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:09.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:09 smithi171 ceph-mon[63127]: pgmap v449: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:09.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:09 smithi129 ceph-mon[62418]: pgmap v449: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:09.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:09 smithi106 ceph-mon[28081]: pgmap v449: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:11.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:11 smithi129 ceph-mon[62418]: pgmap v450: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:11.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:11 smithi106 ceph-mon[28081]: pgmap v450: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:11.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:11 smithi171 ceph-mon[63127]: pgmap v450: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:13.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:13 smithi129 ceph-mon[62418]: pgmap v451: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:13.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:13 smithi106 ceph-mon[28081]: pgmap v451: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:13.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:13 smithi171 ceph-mon[63127]: pgmap v451: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:15.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:15 smithi129 ceph-mon[62418]: pgmap v452: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:15.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:15 smithi106 ceph-mon[28081]: pgmap v452: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:15.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:15 smithi171 ceph-mon[63127]: pgmap v452: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:16.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:16 smithi106 ceph-mon[28081]: pgmap v453: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:16.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:16 smithi171 ceph-mon[63127]: pgmap v453: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:16.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:16 smithi129 ceph-mon[62418]: pgmap v453: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:19.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:19 smithi171 ceph-mon[63127]: pgmap v454: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:19.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:19 smithi129 ceph-mon[62418]: pgmap v454: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:19.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:19 smithi106 ceph-mon[28081]: pgmap v454: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:21.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:21 smithi171 ceph-mon[63127]: pgmap v455: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:21.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:21 smithi129 ceph-mon[62418]: pgmap v455: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:21.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:21 smithi106 ceph-mon[28081]: pgmap v455: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:23.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:23 smithi171 ceph-mon[63127]: pgmap v456: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:23.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:23 smithi129 ceph-mon[62418]: pgmap v456: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:23.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:23 smithi106 ceph-mon[28081]: pgmap v456: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:25.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:25 smithi171 ceph-mon[63127]: pgmap v457: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:25.514 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:25 smithi129 ceph-mon[62418]: pgmap v457: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:25.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:25 smithi106 ceph-mon[28081]: pgmap v457: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:27.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:27 smithi129 ceph-mon[62418]: pgmap v458: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:27.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:27 smithi106 ceph-mon[28081]: pgmap v458: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:27.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:27 smithi171 ceph-mon[63127]: pgmap v458: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:29.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:29 smithi129 ceph-mon[62418]: pgmap v459: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:29.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:29 smithi106 ceph-mon[28081]: pgmap v459: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:29.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:29 smithi171 ceph-mon[63127]: pgmap v459: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:31.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:31 smithi129 ceph-mon[62418]: pgmap v460: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:31.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:31 smithi106 ceph-mon[28081]: pgmap v460: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:31.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:31 smithi171 ceph-mon[63127]: pgmap v460: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:33.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:33 smithi106 ceph-mon[28081]: pgmap v461: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:33.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:33 smithi171 ceph-mon[63127]: pgmap v461: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:33.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:33 smithi129 ceph-mon[62418]: pgmap v461: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:34.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:34 smithi106 ceph-mon[28081]: pgmap v462: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:34.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:34 smithi171 ceph-mon[63127]: pgmap v462: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:34.764 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:34 smithi129 ceph-mon[62418]: pgmap v462: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:37.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:37 smithi171 ceph-mon[63127]: pgmap v463: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:37.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:37 smithi129 ceph-mon[62418]: pgmap v463: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:37.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:37 smithi106 ceph-mon[28081]: pgmap v463: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T12:53:39.496 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:39 smithi171 ceph-mon[63127]: pgmap v464: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:39.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:39 smithi129 ceph-mon[62418]: pgmap v464: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:39.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:39 smithi106 ceph-mon[28081]: pgmap v464: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:41.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:41 smithi129 ceph-mon[62418]: pgmap v465: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:41.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:41 smithi106 ceph-mon[28081]: pgmap v465: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:41.746 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:41 smithi171 ceph-mon[63127]: pgmap v465: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:43.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:43 smithi129 ceph-mon[62418]: pgmap v466: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:43.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:43 smithi106 ceph-mon[28081]: pgmap v466: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:43.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:43 smithi171 ceph-mon[63127]: pgmap v466: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:45.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:45 smithi129 ceph-mon[62418]: pgmap v467: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:45.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:45 smithi106 ceph-mon[28081]: pgmap v467: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:45.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:45 smithi171 ceph-mon[63127]: pgmap v467: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:47.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:47 smithi106 ceph-mon[28081]: pgmap v468: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:47.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:47 smithi171 ceph-mon[63127]: pgmap v468: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:47.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:47 smithi129 ceph-mon[62418]: pgmap v468: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:49.567 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:49 smithi129 ceph-mon[62418]: pgmap v469: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:49.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:49 smithi106 ceph-mon[28081]: pgmap v469: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:49.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:49 smithi171 ceph-mon[63127]: pgmap v469: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-28T12:53:50.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:50 smithi106 ceph-mon[28081]: pgmap v470: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:50.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:50 smithi171 ceph-mon[63127]: pgmap v470: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:50.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:50 smithi129 ceph-mon[62418]: pgmap v470: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:53:53.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:53 smithi129 ceph-mon[62418]: pgmap v471: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:53.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:53 smithi106 ceph-mon[28081]: pgmap v471: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:53.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:53 smithi171 ceph-mon[63127]: pgmap v471: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:55.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:55 smithi129 ceph-mon[62418]: pgmap v472: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:55.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:55 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:53:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:55 smithi106 ceph-mon[28081]: pgmap v472: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:55 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:53:55.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:55 smithi171 ceph-mon[63127]: pgmap v472: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:53:55.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:55 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:53:56.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:53:56.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:53:56.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:53:56.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:53:56.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:53:56.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:53:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:53:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:53:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:53:57.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:57 smithi129 ceph-mon[62418]: pgmap v473: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:57.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:57 smithi106 ceph-mon[28081]: pgmap v473: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:57.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:57 smithi171 ceph-mon[63127]: pgmap v473: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:53:59.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:53:59 smithi129 ceph-mon[62418]: pgmap v474: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:53:59.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:53:59 smithi106 ceph-mon[28081]: pgmap v474: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:53:59.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:53:59 smithi171 ceph-mon[63127]: pgmap v474: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:01.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:01 smithi106 ceph-mon[28081]: pgmap v475: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:01.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:01 smithi171 ceph-mon[63127]: pgmap v475: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:01.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:01 smithi129 ceph-mon[62418]: pgmap v475: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:03.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:03 smithi106 ceph-mon[28081]: pgmap v476: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:03.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:03 smithi171 ceph-mon[63127]: pgmap v476: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:03.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:03 smithi129 ceph-mon[62418]: pgmap v476: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:04.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:04 smithi106 ceph-mon[28081]: pgmap v477: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:04.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:04 smithi171 ceph-mon[63127]: pgmap v477: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:04.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:04 smithi129 ceph-mon[62418]: pgmap v477: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:07.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:07 smithi129 ceph-mon[62418]: pgmap v478: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:07.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:07 smithi106 ceph-mon[28081]: pgmap v478: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:07.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:07 smithi171 ceph-mon[63127]: pgmap v478: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:09.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:09 smithi129 ceph-mon[62418]: pgmap v479: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:09.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:09 smithi106 ceph-mon[28081]: pgmap v479: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:09.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:09 smithi171 ceph-mon[63127]: pgmap v479: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:11.549 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:11 smithi171 ceph-mon[63127]: pgmap v480: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:11.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:11 smithi106 ceph-mon[28081]: pgmap v480: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:11.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:11 smithi129 ceph-mon[62418]: pgmap v480: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:13.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:13 smithi106 ceph-mon[28081]: pgmap v481: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:13.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:13 smithi171 ceph-mon[63127]: pgmap v481: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:13.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:13 smithi129 ceph-mon[62418]: pgmap v481: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:14.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:14 smithi171 ceph-mon[63127]: pgmap v482: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:14.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:14 smithi129 ceph-mon[62418]: pgmap v482: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:14.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:14 smithi106 ceph-mon[28081]: pgmap v482: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:17.515 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:17 smithi129 ceph-mon[62418]: pgmap v483: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:17.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:17 smithi106 ceph-mon[28081]: pgmap v483: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:17.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:17 smithi171 ceph-mon[63127]: pgmap v483: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:19.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:19 smithi106 ceph-mon[28081]: pgmap v484: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:19.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:19 smithi171 ceph-mon[63127]: pgmap v484: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:19.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:19 smithi129 ceph-mon[62418]: pgmap v484: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:21.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:21 smithi106 ceph-mon[28081]: pgmap v485: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:21.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:21 smithi171 ceph-mon[63127]: pgmap v485: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:21.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:21 smithi129 ceph-mon[62418]: pgmap v485: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:23.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:23 smithi106 ceph-mon[28081]: pgmap v486: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:23.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:23 smithi171 ceph-mon[63127]: pgmap v486: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:23.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:23 smithi129 ceph-mon[62418]: pgmap v486: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:24.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:24 smithi171 ceph-mon[63127]: pgmap v487: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:24.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:24 smithi129 ceph-mon[62418]: pgmap v487: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:24.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:24 smithi106 ceph-mon[28081]: pgmap v487: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:27.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:27 smithi106 ceph-mon[28081]: pgmap v488: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:27 smithi171 ceph-mon[63127]: pgmap v488: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:27.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:27 smithi129 ceph-mon[62418]: pgmap v488: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:29.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:29 smithi106 ceph-mon[28081]: pgmap v489: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:29.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:29 smithi171 ceph-mon[63127]: pgmap v489: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:29.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:29 smithi129 ceph-mon[62418]: pgmap v489: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:31.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:31 smithi106 ceph-mon[28081]: pgmap v490: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:31.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:31 smithi171 ceph-mon[63127]: pgmap v490: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:31.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:31 smithi129 ceph-mon[62418]: pgmap v490: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:32.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:32 smithi171 ceph-mon[63127]: pgmap v491: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:32.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:32 smithi129 ceph-mon[62418]: pgmap v491: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:32.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:32 smithi106 ceph-mon[28081]: pgmap v491: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:35.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:35 smithi106 ceph-mon[28081]: pgmap v492: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:35.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:35 smithi171 ceph-mon[63127]: pgmap v492: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:35.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:35 smithi129 ceph-mon[62418]: pgmap v492: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:37.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:37 smithi106 ceph-mon[28081]: pgmap v493: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:37.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:37 smithi171 ceph-mon[63127]: pgmap v493: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:37.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:37 smithi129 ceph-mon[62418]: pgmap v493: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:39.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:39 smithi106 ceph-mon[28081]: pgmap v494: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:39 smithi171 ceph-mon[63127]: pgmap v494: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:39.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:39 smithi129 ceph-mon[62418]: pgmap v494: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:40.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:40 smithi106 ceph-mon[28081]: pgmap v495: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:40 smithi171 ceph-mon[63127]: pgmap v495: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:40.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:40 smithi129 ceph-mon[62418]: pgmap v495: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:43.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:43 smithi106 ceph-mon[28081]: pgmap v496: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:43.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:43 smithi171 ceph-mon[63127]: pgmap v496: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:43.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:43 smithi129 ceph-mon[62418]: pgmap v496: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:45.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:45 smithi106 ceph-mon[28081]: pgmap v497: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:45.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:45 smithi171 ceph-mon[63127]: pgmap v497: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:45.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:45 smithi129 ceph-mon[62418]: pgmap v497: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:46.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:46 smithi171 ceph-mon[63127]: pgmap v498: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:46.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:46 smithi129 ceph-mon[62418]: pgmap v498: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:46.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:46 smithi106 ceph-mon[28081]: pgmap v498: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:47.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:54:47.447+0000 7fe054c00640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe0300034c0 con 0x7fe058108fc0 2024-04-28T12:54:47.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:54:47.448+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe0300034c0 con 0x7fe058108fc0 2024-04-28T12:54:49.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:49 smithi106 ceph-mon[28081]: pgmap v499: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:49.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:49 smithi171 ceph-mon[63127]: pgmap v499: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:49.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:49 smithi129 ceph-mon[62418]: pgmap v499: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:51.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:51 smithi106 ceph-mon[28081]: pgmap v500: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:51.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:51 smithi171 ceph-mon[63127]: pgmap v500: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:51.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:51 smithi129 ceph-mon[62418]: pgmap v500: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:54:52.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:52 smithi106 ceph-mon[28081]: pgmap v501: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:52.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:52 smithi171 ceph-mon[63127]: pgmap v501: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:52.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:52 smithi129 ceph-mon[62418]: pgmap v501: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:55.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:55 smithi106 ceph-mon[28081]: pgmap v502: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:55.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:55 smithi171 ceph-mon[63127]: pgmap v502: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:55.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:55 smithi129 ceph-mon[62418]: pgmap v502: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:54:56.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:56 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:54:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:56 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:54:56.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:56 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:54:57.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:57 smithi106 ceph-mon[28081]: pgmap v503: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:57.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:57 smithi171 ceph-mon[63127]: pgmap v503: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:57.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:57 smithi129 ceph-mon[62418]: pgmap v503: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:54:58.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:54:58.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:54:58.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:58 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:54:58.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:54:58 smithi106 ceph-mon[28081]: pgmap v504: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:54:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:54:58.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:58 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:54:58.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:54:58 smithi171 ceph-mon[63127]: pgmap v504: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:54:58.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:54:58.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:54:58.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:58 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:54:58.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:54:58 smithi129 ceph-mon[62418]: pgmap v504: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:01.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:01 smithi106 ceph-mon[28081]: pgmap v505: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:01.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:01 smithi171 ceph-mon[63127]: pgmap v505: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:01.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:01 smithi129 ceph-mon[62418]: pgmap v505: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:02.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:02 smithi171 ceph-mon[63127]: pgmap v506: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:02.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:02 smithi129 ceph-mon[62418]: pgmap v506: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:02.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:02 smithi106 ceph-mon[28081]: pgmap v506: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:05.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:05 smithi106 ceph-mon[28081]: pgmap v507: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:05.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:05 smithi171 ceph-mon[63127]: pgmap v507: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:05.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:05 smithi129 ceph-mon[62418]: pgmap v507: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:06.576 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:06 smithi106 ceph-mon[28081]: pgmap v508: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:06.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:06 smithi171 ceph-mon[63127]: pgmap v508: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:06 smithi129 ceph-mon[62418]: pgmap v508: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:08.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:08 smithi171 ceph-mon[63127]: pgmap v509: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:08.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:08 smithi129 ceph-mon[62418]: pgmap v509: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:08.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:08 smithi106 ceph-mon[28081]: pgmap v509: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:10.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:10 smithi171 ceph-mon[63127]: pgmap v510: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:10.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:10 smithi129 ceph-mon[62418]: pgmap v510: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:10.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:10 smithi106 ceph-mon[28081]: pgmap v510: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:12.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:12 smithi171 ceph-mon[63127]: pgmap v511: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:12.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:12 smithi129 ceph-mon[62418]: pgmap v511: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:12.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:12 smithi106 ceph-mon[28081]: pgmap v511: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:14.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:14 smithi171 ceph-mon[63127]: pgmap v512: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:14.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:14 smithi129 ceph-mon[62418]: pgmap v512: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:14.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:14 smithi106 ceph-mon[28081]: pgmap v512: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:16.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:16 smithi171 ceph-mon[63127]: pgmap v513: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:16.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:16 smithi129 ceph-mon[62418]: pgmap v513: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:16.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:16 smithi106 ceph-mon[28081]: pgmap v513: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:18.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:18 smithi171 ceph-mon[63127]: pgmap v514: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:18.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:18 smithi129 ceph-mon[62418]: pgmap v514: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:18.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:18 smithi106 ceph-mon[28081]: pgmap v514: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:20.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:20 smithi171 ceph-mon[63127]: pgmap v515: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:20.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:20 smithi129 ceph-mon[62418]: pgmap v515: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:20.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:20 smithi106 ceph-mon[28081]: pgmap v515: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:22.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:22 smithi171 ceph-mon[63127]: pgmap v516: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:22.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:22 smithi129 ceph-mon[62418]: pgmap v516: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:22.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:22 smithi106 ceph-mon[28081]: pgmap v516: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:24.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:24 smithi171 ceph-mon[63127]: pgmap v517: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:55:24.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:24 smithi129 ceph-mon[62418]: pgmap v517: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:55:24.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:24 smithi106 ceph-mon[28081]: pgmap v517: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:55:26.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:26 smithi171 ceph-mon[63127]: pgmap v518: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:55:26.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:26 smithi129 ceph-mon[62418]: pgmap v518: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:55:26.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:26 smithi106 ceph-mon[28081]: pgmap v518: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:55:28.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:28 smithi171 ceph-mon[63127]: pgmap v519: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-28T12:55:28.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:28 smithi129 ceph-mon[62418]: pgmap v519: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-28T12:55:28.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:28 smithi106 ceph-mon[28081]: pgmap v519: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-28T12:55:30.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:30 smithi171 ceph-mon[63127]: pgmap v520: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:55:30.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:30 smithi129 ceph-mon[62418]: pgmap v520: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:55:30.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:30 smithi106 ceph-mon[28081]: pgmap v520: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:55:32.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:32 smithi171 ceph-mon[63127]: pgmap v521: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:55:32.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:32 smithi129 ceph-mon[62418]: pgmap v521: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:55:32.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:32 smithi106 ceph-mon[28081]: pgmap v521: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:55:34.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:34 smithi171 ceph-mon[63127]: pgmap v522: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:55:34.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:34 smithi129 ceph-mon[62418]: pgmap v522: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:55:34.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:34 smithi106 ceph-mon[28081]: pgmap v522: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:55:36.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:36 smithi171 ceph-mon[63127]: pgmap v523: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:36.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:36 smithi129 ceph-mon[62418]: pgmap v523: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:36.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:36 smithi106 ceph-mon[28081]: pgmap v523: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:38.630 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:38 smithi106 ceph-mon[28081]: pgmap v524: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:38.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:38 smithi171 ceph-mon[63127]: pgmap v524: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:38.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:38 smithi129 ceph-mon[62418]: pgmap v524: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:40 smithi171 ceph-mon[63127]: pgmap v525: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:40.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:40 smithi129 ceph-mon[62418]: pgmap v525: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:40.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:40 smithi106 ceph-mon[28081]: pgmap v525: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:42.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:42 smithi171 ceph-mon[63127]: pgmap v526: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:42.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:42 smithi129 ceph-mon[62418]: pgmap v526: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:42.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:42 smithi106 ceph-mon[28081]: pgmap v526: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:44 smithi171 ceph-mon[63127]: pgmap v527: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:44.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:44 smithi129 ceph-mon[62418]: pgmap v527: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:44.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:44 smithi106 ceph-mon[28081]: pgmap v527: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:46.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:46 smithi171 ceph-mon[63127]: pgmap v528: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:46.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:46 smithi129 ceph-mon[62418]: pgmap v528: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:46.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:46 smithi106 ceph-mon[28081]: pgmap v528: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:48.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:48 smithi171 ceph-mon[63127]: pgmap v529: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:48.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:48 smithi129 ceph-mon[62418]: pgmap v529: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:48.826 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:48 smithi106 ceph-mon[28081]: pgmap v529: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:50.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:50 smithi171 ceph-mon[63127]: pgmap v530: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:50.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:50 smithi129 ceph-mon[62418]: pgmap v530: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:50.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:50 smithi106 ceph-mon[28081]: pgmap v530: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:55:52.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:52 smithi171 ceph-mon[63127]: pgmap v531: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:52.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:52 smithi129 ceph-mon[62418]: pgmap v531: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:52.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:52 smithi106 ceph-mon[28081]: pgmap v531: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:54.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:54 smithi171 ceph-mon[63127]: pgmap v532: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:54.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:54 smithi129 ceph-mon[62418]: pgmap v532: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:54.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:54 smithi106 ceph-mon[28081]: pgmap v532: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:55:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:56 smithi171 ceph-mon[63127]: pgmap v533: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:56.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:56 smithi129 ceph-mon[62418]: pgmap v533: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:56.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:56 smithi106 ceph-mon[28081]: pgmap v533: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:55:58.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:57 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:55:58.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:57 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:55:58.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:57 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:55:59.076 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:58 smithi106 ceph-mon[28081]: pgmap v534: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:58 smithi171 ceph-mon[63127]: pgmap v534: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:55:59.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:58 smithi129 ceph-mon[62418]: pgmap v534: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:00.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:56:00.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:56:00.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:55:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:56:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:56:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:56:00.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:55:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:56:00.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:56:00.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:56:00.265 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:55:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:56:01.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:00 smithi106 ceph-mon[28081]: pgmap v535: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:00 smithi171 ceph-mon[63127]: pgmap v535: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:01.266 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:00 smithi129 ceph-mon[62418]: pgmap v535: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:02.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:02 smithi171 ceph-mon[63127]: pgmap v536: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:02.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:02 smithi129 ceph-mon[62418]: pgmap v536: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:02.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:02 smithi106 ceph-mon[28081]: pgmap v536: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:04.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:04 smithi171 ceph-mon[63127]: pgmap v537: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:04.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:04 smithi129 ceph-mon[62418]: pgmap v537: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:04 smithi106 ceph-mon[28081]: pgmap v537: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:06.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:06 smithi171 ceph-mon[63127]: pgmap v538: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:06.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:06 smithi129 ceph-mon[62418]: pgmap v538: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:06.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:06 smithi106 ceph-mon[28081]: pgmap v538: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:08.630 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:08 smithi106 ceph-mon[28081]: pgmap v539: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:08.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:08 smithi171 ceph-mon[63127]: pgmap v539: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:08.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:08 smithi129 ceph-mon[62418]: pgmap v539: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:10.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:10 smithi171 ceph-mon[63127]: pgmap v540: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:10.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:10 smithi129 ceph-mon[62418]: pgmap v540: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:10.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:10 smithi106 ceph-mon[28081]: pgmap v540: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:12.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:12 smithi171 ceph-mon[63127]: pgmap v541: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:12.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:12 smithi129 ceph-mon[62418]: pgmap v541: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:12.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:12 smithi106 ceph-mon[28081]: pgmap v541: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:14.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:14 smithi171 ceph-mon[63127]: pgmap v542: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:14.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:14 smithi129 ceph-mon[62418]: pgmap v542: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:14.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:14 smithi106 ceph-mon[28081]: pgmap v542: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:16.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:16 smithi171 ceph-mon[63127]: pgmap v543: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:16.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:16 smithi129 ceph-mon[62418]: pgmap v543: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:16.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:16 smithi106 ceph-mon[28081]: pgmap v543: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:18.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:18 smithi171 ceph-mon[63127]: pgmap v544: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:18.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:18 smithi129 ceph-mon[62418]: pgmap v544: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:18.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:18 smithi106 ceph-mon[28081]: pgmap v544: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:20.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:20 smithi171 ceph-mon[63127]: pgmap v545: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:20.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:20 smithi129 ceph-mon[62418]: pgmap v545: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:20.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:20 smithi106 ceph-mon[28081]: pgmap v545: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:22.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:22 smithi171 ceph-mon[63127]: pgmap v546: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:22.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:22 smithi129 ceph-mon[62418]: pgmap v546: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:22.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:22 smithi106 ceph-mon[28081]: pgmap v546: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:24.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:24 smithi171 ceph-mon[63127]: pgmap v547: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:24.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:24 smithi129 ceph-mon[62418]: pgmap v547: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:24.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:24 smithi106 ceph-mon[28081]: pgmap v547: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:26.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:26 smithi171 ceph-mon[63127]: pgmap v548: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:26.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:26 smithi129 ceph-mon[62418]: pgmap v548: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:26.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:26 smithi106 ceph-mon[28081]: pgmap v548: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:28.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:28 smithi171 ceph-mon[63127]: pgmap v549: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:28.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:28 smithi129 ceph-mon[62418]: pgmap v549: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:28.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:28 smithi106 ceph-mon[28081]: pgmap v549: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:30.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:30 smithi171 ceph-mon[63127]: pgmap v550: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:30.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:30 smithi129 ceph-mon[62418]: pgmap v550: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:30.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:30 smithi106 ceph-mon[28081]: pgmap v550: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:32.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:32 smithi171 ceph-mon[63127]: pgmap v551: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:32.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:32 smithi129 ceph-mon[62418]: pgmap v551: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:32.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:32 smithi106 ceph-mon[28081]: pgmap v551: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:34.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:34 smithi171 ceph-mon[63127]: pgmap v552: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:34.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:34 smithi129 ceph-mon[62418]: pgmap v552: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:34.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:34 smithi106 ceph-mon[28081]: pgmap v552: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:36.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:36 smithi171 ceph-mon[63127]: pgmap v553: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:36.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:36 smithi129 ceph-mon[62418]: pgmap v553: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:36.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:36 smithi106 ceph-mon[28081]: pgmap v553: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:38.630 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:38 smithi106 ceph-mon[28081]: pgmap v554: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:38.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:38 smithi171 ceph-mon[63127]: pgmap v554: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:38.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:38 smithi129 ceph-mon[62418]: pgmap v554: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:40 smithi171 ceph-mon[63127]: pgmap v555: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:40.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:40 smithi129 ceph-mon[62418]: pgmap v555: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:40.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:40 smithi106 ceph-mon[28081]: pgmap v555: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:42.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:42 smithi171 ceph-mon[63127]: pgmap v556: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:42.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:42 smithi129 ceph-mon[62418]: pgmap v556: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:42.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:42 smithi106 ceph-mon[28081]: pgmap v556: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:44 smithi171 ceph-mon[63127]: pgmap v557: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:44.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:44 smithi129 ceph-mon[62418]: pgmap v557: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:44.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:44 smithi106 ceph-mon[28081]: pgmap v557: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:46 smithi171 ceph-mon[63127]: pgmap v558: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:46.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:46 smithi129 ceph-mon[62418]: pgmap v558: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:46.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:46 smithi106 ceph-mon[28081]: pgmap v558: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:48.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:48 smithi171 ceph-mon[63127]: pgmap v559: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:48.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:48 smithi129 ceph-mon[62418]: pgmap v559: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:48.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:48 smithi106 ceph-mon[28081]: pgmap v559: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:50.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:50 smithi171 ceph-mon[63127]: pgmap v560: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:50.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:50 smithi129 ceph-mon[62418]: pgmap v560: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:50.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:50 smithi106 ceph-mon[28081]: pgmap v560: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:56:52.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:52 smithi171 ceph-mon[63127]: pgmap v561: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:52.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:52 smithi129 ceph-mon[62418]: pgmap v561: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:52.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:52 smithi106 ceph-mon[28081]: pgmap v561: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:54.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:54 smithi171 ceph-mon[63127]: pgmap v562: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:54.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:54 smithi129 ceph-mon[62418]: pgmap v562: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:54.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:54 smithi106 ceph-mon[28081]: pgmap v562: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:56:56.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:56 smithi171 ceph-mon[63127]: pgmap v563: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:56.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:56 smithi129 ceph-mon[62418]: pgmap v563: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:56.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:56 smithi106 ceph-mon[28081]: pgmap v563: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:56:58.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:58 smithi171 ceph-mon[63127]: pgmap v564: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:58.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:58 smithi129 ceph-mon[62418]: pgmap v564: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:58.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:58 smithi106 ceph-mon[28081]: pgmap v564: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:56:59.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:56:59 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:56:59.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:56:59 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:56:59.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:56:59 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:57:00.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:00 smithi171 ceph-mon[63127]: pgmap v565: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:00.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:00 smithi129 ceph-mon[62418]: pgmap v565: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:00.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:00 smithi106 ceph-mon[28081]: pgmap v565: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:01.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:57:01.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:57:01.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:57:01.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:57:01.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:57:01.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:57:01.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:57:01.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:57:01.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:57:02.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:02 smithi171 ceph-mon[63127]: pgmap v566: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:02.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:02 smithi129 ceph-mon[62418]: pgmap v566: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:02.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:02 smithi106 ceph-mon[28081]: pgmap v566: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:04.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:04 smithi171 ceph-mon[63127]: pgmap v567: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:04.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:04 smithi129 ceph-mon[62418]: pgmap v567: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:04.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:04 smithi106 ceph-mon[28081]: pgmap v567: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:06.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:06 smithi171 ceph-mon[63127]: pgmap v568: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:06.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:06 smithi129 ceph-mon[62418]: pgmap v568: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:06.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:06 smithi106 ceph-mon[28081]: pgmap v568: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:08.630 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:08 smithi106 ceph-mon[28081]: pgmap v569: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:08.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:08 smithi171 ceph-mon[63127]: pgmap v569: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:08.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:08 smithi129 ceph-mon[62418]: pgmap v569: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:10.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:10 smithi171 ceph-mon[63127]: pgmap v570: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:10.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:10 smithi129 ceph-mon[62418]: pgmap v570: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:10.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:10 smithi106 ceph-mon[28081]: pgmap v570: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:12.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:12 smithi171 ceph-mon[63127]: pgmap v571: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:12.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:12 smithi129 ceph-mon[62418]: pgmap v571: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:12.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:12 smithi106 ceph-mon[28081]: pgmap v571: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:14.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:14 smithi171 ceph-mon[63127]: pgmap v572: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:14.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:14 smithi129 ceph-mon[62418]: pgmap v572: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:14.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:14 smithi106 ceph-mon[28081]: pgmap v572: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:16.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:16 smithi171 ceph-mon[63127]: pgmap v573: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:16.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:16 smithi129 ceph-mon[62418]: pgmap v573: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:16.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:16 smithi106 ceph-mon[28081]: pgmap v573: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:18.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:18 smithi171 ceph-mon[63127]: pgmap v574: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:18.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:18 smithi129 ceph-mon[62418]: pgmap v574: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:18.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:18 smithi106 ceph-mon[28081]: pgmap v574: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:20.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:20 smithi171 ceph-mon[63127]: pgmap v575: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:20.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:20 smithi129 ceph-mon[62418]: pgmap v575: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:20.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:20 smithi106 ceph-mon[28081]: pgmap v575: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:22.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:22 smithi171 ceph-mon[63127]: pgmap v576: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:22.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:22 smithi129 ceph-mon[62418]: pgmap v576: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:22.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:22 smithi106 ceph-mon[28081]: pgmap v576: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:24.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:24 smithi171 ceph-mon[63127]: pgmap v577: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:24.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:24 smithi129 ceph-mon[62418]: pgmap v577: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:24.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:24 smithi106 ceph-mon[28081]: pgmap v577: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:26.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:26 smithi171 ceph-mon[63127]: pgmap v578: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:26.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:26 smithi129 ceph-mon[62418]: pgmap v578: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:26.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:26 smithi106 ceph-mon[28081]: pgmap v578: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:28.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:28 smithi171 ceph-mon[63127]: pgmap v579: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:28.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:28 smithi129 ceph-mon[62418]: pgmap v579: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:28.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:28 smithi106 ceph-mon[28081]: pgmap v579: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:30.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:30 smithi171 ceph-mon[63127]: pgmap v580: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:30.765 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:30 smithi129 ceph-mon[62418]: pgmap v580: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:30.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:30 smithi106 ceph-mon[28081]: pgmap v580: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:32.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:32 smithi171 ceph-mon[63127]: pgmap v581: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:32.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:32 smithi129 ceph-mon[62418]: pgmap v581: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:32.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:32 smithi106 ceph-mon[28081]: pgmap v581: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:34.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:34 smithi171 ceph-mon[63127]: pgmap v582: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:34.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:34 smithi129 ceph-mon[62418]: pgmap v582: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:34.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:34 smithi106 ceph-mon[28081]: pgmap v582: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:36.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:36 smithi171 ceph-mon[63127]: pgmap v583: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:36.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:36 smithi129 ceph-mon[62418]: pgmap v583: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:36.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:36 smithi106 ceph-mon[28081]: pgmap v583: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:38.630 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:38 smithi106 ceph-mon[28081]: pgmap v584: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:38.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:38 smithi171 ceph-mon[63127]: pgmap v584: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:38.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:38 smithi129 ceph-mon[62418]: pgmap v584: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:40.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:40 smithi171 ceph-mon[63127]: pgmap v585: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:40.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:40 smithi129 ceph-mon[62418]: pgmap v585: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:40.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:40 smithi106 ceph-mon[28081]: pgmap v585: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:42.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:42 smithi171 ceph-mon[63127]: pgmap v586: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:42.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:42 smithi129 ceph-mon[62418]: pgmap v586: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:42.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:42 smithi106 ceph-mon[28081]: pgmap v586: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:44.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:44 smithi171 ceph-mon[63127]: pgmap v587: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:44.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:44 smithi129 ceph-mon[62418]: pgmap v587: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:44.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:44 smithi106 ceph-mon[28081]: pgmap v587: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:46 smithi171 ceph-mon[63127]: pgmap v588: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:46.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:46 smithi129 ceph-mon[62418]: pgmap v588: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:46.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:46 smithi106 ceph-mon[28081]: pgmap v588: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:47.475 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:57:47.476+0000 7fe054c00640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe030005850 con 0x7fe058108fc0 2024-04-28T12:57:47.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T12:57:47.477+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe030005850 con 0x7fe058108fc0 2024-04-28T12:57:48.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:48 smithi171 ceph-mon[63127]: pgmap v589: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:48.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:48 smithi129 ceph-mon[62418]: pgmap v589: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:48.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:48 smithi106 ceph-mon[28081]: pgmap v589: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:50.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:50 smithi171 ceph-mon[63127]: pgmap v590: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:50.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:50 smithi129 ceph-mon[62418]: pgmap v590: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:50.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:50 smithi106 ceph-mon[28081]: pgmap v590: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:57:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:52 smithi171 ceph-mon[63127]: pgmap v591: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:52.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:52 smithi129 ceph-mon[62418]: pgmap v591: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:52.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:52 smithi106 ceph-mon[28081]: pgmap v591: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:54.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:54 smithi171 ceph-mon[63127]: pgmap v592: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:54.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:54 smithi129 ceph-mon[62418]: pgmap v592: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:54.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:54 smithi106 ceph-mon[28081]: pgmap v592: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:57:57.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:57 smithi171 ceph-mon[63127]: pgmap v593: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:57.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:57 smithi129 ceph-mon[62418]: pgmap v593: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:57.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:57 smithi106 ceph-mon[28081]: pgmap v593: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:57:59.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:57:59 smithi171 ceph-mon[63127]: pgmap v594: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:57:59 smithi129 ceph-mon[62418]: pgmap v594: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:57:59.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:57:59 smithi106 ceph-mon[28081]: pgmap v594: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:01.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:01 smithi171 ceph-mon[63127]: pgmap v595: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:01.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:01 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:58:01.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:01 smithi129 ceph-mon[62418]: pgmap v595: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:01.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:01 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:58:01.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:01 smithi106 ceph-mon[28081]: pgmap v595: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:01.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:01 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:58:02.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi129 to 176.2M 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: Unable to set osd_memory_target on smithi129 to 184782438: error parsing value: Value '184782438' is below minimum 939524096 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: Unable to set osd_memory_target on smithi171 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: Adjusting osd_memory_target on smithi106 to 176.4M 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: Unable to set osd_memory_target on smithi106 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:58:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:02 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:58:02.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi129 to 176.2M 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: Unable to set osd_memory_target on smithi129 to 184782438: error parsing value: Value '184782438' is below minimum 939524096 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: Unable to set osd_memory_target on smithi171 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: Adjusting osd_memory_target on smithi106 to 176.4M 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: Unable to set osd_memory_target on smithi106 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:58:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:02 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:58:02.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi129 to 176.2M 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi171 to 176.2M 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: Unable to set osd_memory_target on smithi129 to 184782438: error parsing value: Value '184782438' is below minimum 939524096 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: Unable to set osd_memory_target on smithi171 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: Adjusting osd_memory_target on smithi106 to 176.4M 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: Unable to set osd_memory_target on smithi106 to 185061990: error parsing value: Value '185061990' is below minimum 939524096 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:58:02.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:02 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:58:04.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:04 smithi171 ceph-mon[63127]: pgmap v596: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:04.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:04 smithi129 ceph-mon[62418]: pgmap v596: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:04.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:04 smithi106 ceph-mon[28081]: pgmap v596: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:06.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:06 smithi171 ceph-mon[63127]: pgmap v597: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:06.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:06 smithi129 ceph-mon[62418]: pgmap v597: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:06.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:06 smithi106 ceph-mon[28081]: pgmap v597: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:08 smithi171 ceph-mon[63127]: pgmap v598: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:08.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:08 smithi129 ceph-mon[62418]: pgmap v598: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:08.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:08 smithi106 ceph-mon[28081]: pgmap v598: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:10 smithi171 ceph-mon[63127]: pgmap v599: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:10.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:10 smithi129 ceph-mon[62418]: pgmap v599: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:10.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:10 smithi106 ceph-mon[28081]: pgmap v599: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:12 smithi171 ceph-mon[63127]: pgmap v600: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:12.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:12 smithi129 ceph-mon[62418]: pgmap v600: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:12.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:12 smithi106 ceph-mon[28081]: pgmap v600: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:14.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:14 smithi171 ceph-mon[63127]: pgmap v601: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:14.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:14 smithi129 ceph-mon[62418]: pgmap v601: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:14.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:14 smithi106 ceph-mon[28081]: pgmap v601: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:16.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:16 smithi171 ceph-mon[63127]: pgmap v602: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:16.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:16 smithi129 ceph-mon[62418]: pgmap v602: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:16.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:16 smithi106 ceph-mon[28081]: pgmap v602: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:18.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:18 smithi171 ceph-mon[63127]: pgmap v603: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:18.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:18 smithi129 ceph-mon[62418]: pgmap v603: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:18.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:18 smithi106 ceph-mon[28081]: pgmap v603: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:20 smithi171 ceph-mon[63127]: pgmap v604: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:20.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:20 smithi129 ceph-mon[62418]: pgmap v604: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:20.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:20 smithi106 ceph-mon[28081]: pgmap v604: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:22.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:22 smithi171 ceph-mon[63127]: pgmap v605: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:22.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:22 smithi129 ceph-mon[62418]: pgmap v605: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:22.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:22 smithi106 ceph-mon[28081]: pgmap v605: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:24.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:24 smithi171 ceph-mon[63127]: pgmap v606: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:58:24.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:24 smithi129 ceph-mon[62418]: pgmap v606: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:58:24.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:24 smithi106 ceph-mon[28081]: pgmap v606: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:58:26.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:26 smithi171 ceph-mon[63127]: pgmap v607: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:58:26.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:26 smithi129 ceph-mon[62418]: pgmap v607: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:58:26.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:26 smithi106 ceph-mon[28081]: pgmap v607: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:58:28.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:28 smithi171 ceph-mon[63127]: pgmap v608: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:58:28.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:28 smithi129 ceph-mon[62418]: pgmap v608: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:58:28.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:28 smithi106 ceph-mon[28081]: pgmap v608: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:58:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:30 smithi171 ceph-mon[63127]: pgmap v609: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-28T12:58:30.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:30 smithi129 ceph-mon[62418]: pgmap v609: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-28T12:58:30.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:30 smithi106 ceph-mon[28081]: pgmap v609: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-28T12:58:32.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:32 smithi171 ceph-mon[63127]: pgmap v610: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:58:32.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:32 smithi129 ceph-mon[62418]: pgmap v610: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:58:32.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:32 smithi106 ceph-mon[28081]: pgmap v610: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:58:34.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:34 smithi171 ceph-mon[63127]: pgmap v611: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:58:34.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:34 smithi129 ceph-mon[62418]: pgmap v611: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:58:34.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:34 smithi106 ceph-mon[28081]: pgmap v611: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:58:36.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:36 smithi171 ceph-mon[63127]: pgmap v612: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:36.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:36 smithi129 ceph-mon[62418]: pgmap v612: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:36.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:36 smithi106 ceph-mon[28081]: pgmap v612: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:38.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:38 smithi171 ceph-mon[63127]: pgmap v613: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:38.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:38 smithi129 ceph-mon[62418]: pgmap v613: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:38.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:38 smithi106 ceph-mon[28081]: pgmap v613: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:40.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:40 smithi171 ceph-mon[63127]: pgmap v614: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:40.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:40 smithi129 ceph-mon[62418]: pgmap v614: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:40.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:40 smithi106 ceph-mon[28081]: pgmap v614: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:42.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:42 smithi171 ceph-mon[63127]: pgmap v615: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:42.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:42 smithi129 ceph-mon[62418]: pgmap v615: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:42.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:42 smithi106 ceph-mon[28081]: pgmap v615: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:44.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:44 smithi171 ceph-mon[63127]: pgmap v616: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:44.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:44 smithi129 ceph-mon[62418]: pgmap v616: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:44.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:44 smithi106 ceph-mon[28081]: pgmap v616: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:46.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:46 smithi171 ceph-mon[63127]: pgmap v617: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:46.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:46 smithi129 ceph-mon[62418]: pgmap v617: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:46.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:46 smithi106 ceph-mon[28081]: pgmap v617: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:48.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:48 smithi129 ceph-mon[62418]: pgmap v618: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:48.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:48 smithi106 ceph-mon[28081]: pgmap v618: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:48.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:48 smithi171 ceph-mon[63127]: pgmap v618: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:50.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:50 smithi129 ceph-mon[62418]: pgmap v619: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:50.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:50 smithi106 ceph-mon[28081]: pgmap v619: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:50.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:50 smithi171 ceph-mon[63127]: pgmap v619: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:58:52.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:52 smithi129 ceph-mon[62418]: pgmap v620: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:52.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:52 smithi106 ceph-mon[28081]: pgmap v620: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:52 smithi171 ceph-mon[63127]: pgmap v620: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:58:54.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:54 smithi129 ceph-mon[62418]: pgmap v621: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:54.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:54 smithi106 ceph-mon[28081]: pgmap v621: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:54.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:54 smithi171 ceph-mon[63127]: pgmap v621: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:56.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:56 smithi106 ceph-mon[28081]: pgmap v622: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:56.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:56 smithi171 ceph-mon[63127]: pgmap v622: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:56.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:56 smithi129 ceph-mon[62418]: pgmap v622: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:58:58.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:58:58 smithi106 ceph-mon[28081]: pgmap v623: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:58.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:58:58 smithi171 ceph-mon[63127]: pgmap v623: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:58:58.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:58:58 smithi129 ceph-mon[62418]: pgmap v623: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:00.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:00 smithi106 ceph-mon[28081]: pgmap v624: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:00.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:00 smithi171 ceph-mon[63127]: pgmap v624: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:00.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:00 smithi129 ceph-mon[62418]: pgmap v624: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:02.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:02 smithi106 ceph-mon[28081]: pgmap v625: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:02.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:02 smithi171 ceph-mon[63127]: pgmap v625: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:02.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:02 smithi129 ceph-mon[62418]: pgmap v625: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:03.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:03 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:59:03.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:03 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:59:03.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:03 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T12:59:04.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:04 smithi106 ceph-mon[28081]: pgmap v626: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:04.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:59:04.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:59:04.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:59:04.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:04 smithi171 ceph-mon[63127]: pgmap v626: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:04.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:59:04.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:59:04.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:59:04.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:04 smithi129 ceph-mon[62418]: pgmap v626: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:04.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:04 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T12:59:04.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:04 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T12:59:04.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:04 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T12:59:06.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:06 smithi106 ceph-mon[28081]: pgmap v627: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:59:06.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:06 smithi171 ceph-mon[63127]: pgmap v627: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:59:06.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:06 smithi129 ceph-mon[62418]: pgmap v627: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:59:08.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:08 smithi106 ceph-mon[28081]: pgmap v628: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:59:08.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:08 smithi171 ceph-mon[63127]: pgmap v628: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:59:08.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:08 smithi129 ceph-mon[62418]: pgmap v628: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:59:10.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:10 smithi106 ceph-mon[28081]: pgmap v629: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-28T12:59:10.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:10 smithi171 ceph-mon[63127]: pgmap v629: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-28T12:59:10.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:10 smithi129 ceph-mon[62418]: pgmap v629: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-28T12:59:12.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:12 smithi106 ceph-mon[28081]: pgmap v630: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:59:12.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:12 smithi171 ceph-mon[63127]: pgmap v630: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:59:12.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:12 smithi129 ceph-mon[62418]: pgmap v630: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-28T12:59:14.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:14 smithi106 ceph-mon[28081]: pgmap v631: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:59:14.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:14 smithi171 ceph-mon[63127]: pgmap v631: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:59:14.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:14 smithi129 ceph-mon[62418]: pgmap v631: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T12:59:16.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:16 smithi171 ceph-mon[63127]: pgmap v632: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:59:16.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:16 smithi129 ceph-mon[62418]: pgmap v632: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:59:16.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:16 smithi106 ceph-mon[28081]: pgmap v632: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T12:59:18.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:18 smithi171 ceph-mon[63127]: pgmap v633: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:18.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:18 smithi129 ceph-mon[62418]: pgmap v633: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:18.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:18 smithi106 ceph-mon[28081]: pgmap v633: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:20.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:20 smithi171 ceph-mon[63127]: pgmap v634: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:20.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:20 smithi129 ceph-mon[62418]: pgmap v634: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:20.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:20 smithi106 ceph-mon[28081]: pgmap v634: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:21.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:21 smithi171 ceph-mon[63127]: pgmap v635: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:21.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:21 smithi129 ceph-mon[62418]: pgmap v635: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:21.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:21 smithi106 ceph-mon[28081]: pgmap v635: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:23.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:23 smithi171 ceph-mon[63127]: pgmap v636: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:23.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:23 smithi129 ceph-mon[62418]: pgmap v636: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:23.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:23 smithi106 ceph-mon[28081]: pgmap v636: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:25.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:25 smithi171 ceph-mon[63127]: pgmap v637: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:25.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:25 smithi129 ceph-mon[62418]: pgmap v637: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:25.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:25 smithi106 ceph-mon[28081]: pgmap v637: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:27.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:27 smithi171 ceph-mon[63127]: pgmap v638: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:27.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:27 smithi129 ceph-mon[62418]: pgmap v638: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:27.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:27 smithi106 ceph-mon[28081]: pgmap v638: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:29 smithi171 ceph-mon[63127]: pgmap v639: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:29.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:29 smithi129 ceph-mon[62418]: pgmap v639: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:29.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:29 smithi106 ceph-mon[28081]: pgmap v639: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:31.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:31 smithi171 ceph-mon[63127]: pgmap v640: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:31.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:31 smithi129 ceph-mon[62418]: pgmap v640: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:31.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:31 smithi106 ceph-mon[28081]: pgmap v640: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:33.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:33 smithi171 ceph-mon[63127]: pgmap v641: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:33.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:33 smithi129 ceph-mon[62418]: pgmap v641: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:33.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:33 smithi106 ceph-mon[28081]: pgmap v641: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:35.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:35 smithi171 ceph-mon[63127]: pgmap v642: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:35.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:35 smithi129 ceph-mon[62418]: pgmap v642: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:35.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:35 smithi106 ceph-mon[28081]: pgmap v642: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:37.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:37 smithi171 ceph-mon[63127]: pgmap v643: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:37.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:37 smithi129 ceph-mon[62418]: pgmap v643: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:37.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:37 smithi106 ceph-mon[28081]: pgmap v643: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:39.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:39 smithi171 ceph-mon[63127]: pgmap v644: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:39.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:39 smithi129 ceph-mon[62418]: pgmap v644: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:39.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:39 smithi106 ceph-mon[28081]: pgmap v644: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:41.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:41 smithi171 ceph-mon[63127]: pgmap v645: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:41.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:41 smithi129 ceph-mon[62418]: pgmap v645: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:41.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:41 smithi106 ceph-mon[28081]: pgmap v645: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:43.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:43 smithi171 ceph-mon[63127]: pgmap v646: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:43.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:43 smithi129 ceph-mon[62418]: pgmap v646: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:43.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:43 smithi106 ceph-mon[28081]: pgmap v646: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:45.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:45 smithi171 ceph-mon[63127]: pgmap v647: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:45.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:45 smithi129 ceph-mon[62418]: pgmap v647: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:45.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:45 smithi106 ceph-mon[28081]: pgmap v647: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:47.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:47 smithi171 ceph-mon[63127]: pgmap v648: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:47.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:47 smithi129 ceph-mon[62418]: pgmap v648: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:47.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:47 smithi106 ceph-mon[28081]: pgmap v648: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:49.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:49 smithi171 ceph-mon[63127]: pgmap v649: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:49.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:49 smithi129 ceph-mon[62418]: pgmap v649: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:49.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:49 smithi106 ceph-mon[28081]: pgmap v649: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T12:59:52.469 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:52 smithi129 ceph-mon[62418]: pgmap v650: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:52.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:52 smithi171 ceph-mon[63127]: pgmap v650: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:52.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:52 smithi106 ceph-mon[28081]: pgmap v650: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T12:59:54.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:54 smithi171 ceph-mon[63127]: pgmap v651: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:54.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:54 smithi129 ceph-mon[62418]: pgmap v651: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:54.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:54 smithi106 ceph-mon[28081]: pgmap v651: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:56 smithi171 ceph-mon[63127]: pgmap v652: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:56.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:56 smithi129 ceph-mon[62418]: pgmap v652: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:56.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:56 smithi106 ceph-mon[28081]: pgmap v652: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T12:59:58.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 12:59:58 smithi171 ceph-mon[63127]: pgmap v653: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:58.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 12:59:58 smithi129 ceph-mon[62418]: pgmap v653: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T12:59:58.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 12:59:58 smithi106 ceph-mon[28081]: pgmap v653: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:00.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:00 smithi171 ceph-mon[63127]: pgmap v654: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:00.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:00 smithi171 ceph-mon[63127]: overall HEALTH_OK 2024-04-28T13:00:00.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:00 smithi129 ceph-mon[62418]: pgmap v654: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:00.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:00 smithi129 ceph-mon[62418]: overall HEALTH_OK 2024-04-28T13:00:00.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:00 smithi106 ceph-mon[28081]: pgmap v654: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:00.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:00 smithi106 ceph-mon[28081]: overall HEALTH_OK 2024-04-28T13:00:02.498 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:00:02.499+0000 7fe054c00640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7fe030005b70 con 0x7fe058108fc0 2024-04-28T13:00:02.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:02 smithi171 ceph-mon[63127]: pgmap v655: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:02.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:00:02.500+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7fe030005b70 con 0x7fe058108fc0 2024-04-28T13:00:02.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:02 smithi129 ceph-mon[62418]: pgmap v655: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:02.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:02 smithi106 ceph-mon[28081]: pgmap v655: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:04.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:04 smithi171 ceph-mon[63127]: pgmap v656: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:04.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:04 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:00:04.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:04 smithi129 ceph-mon[62418]: pgmap v656: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:04.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:04 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:00:04.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:04 smithi106 ceph-mon[28081]: pgmap v656: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:04.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:04 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:00:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:05 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:00:05.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:05 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:00:05.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:05 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:00:05.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:05 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:00:05.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:05 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:00:05.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:05 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:00:05.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:05 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:00:05.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:05 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:00:05.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:05 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:00:06.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:06 smithi171 ceph-mon[63127]: pgmap v657: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:06.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:06 smithi129 ceph-mon[62418]: pgmap v657: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:06.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:06 smithi106 ceph-mon[28081]: pgmap v657: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:08.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:08 smithi129 ceph-mon[62418]: pgmap v658: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:08.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:08 smithi106 ceph-mon[28081]: pgmap v658: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:08.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:08 smithi171 ceph-mon[63127]: pgmap v658: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:10.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:10 smithi129 ceph-mon[62418]: pgmap v659: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:10.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:10 smithi106 ceph-mon[28081]: pgmap v659: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:10.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:10 smithi171 ceph-mon[63127]: pgmap v659: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:12.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:12 smithi129 ceph-mon[62418]: pgmap v660: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:12.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:12 smithi106 ceph-mon[28081]: pgmap v660: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:12.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:12 smithi171 ceph-mon[63127]: pgmap v660: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:14.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:14 smithi106 ceph-mon[28081]: pgmap v661: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:14.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:14 smithi171 ceph-mon[63127]: pgmap v661: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:14.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:14 smithi129 ceph-mon[62418]: pgmap v661: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:16.551 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:16 smithi171 ceph-mon[63127]: pgmap v662: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:16.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:16 smithi106 ceph-mon[28081]: pgmap v662: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:16.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:16 smithi129 ceph-mon[62418]: pgmap v662: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:18.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:18 smithi106 ceph-mon[28081]: pgmap v663: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:18.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:18 smithi171 ceph-mon[63127]: pgmap v663: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:18.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:18 smithi129 ceph-mon[62418]: pgmap v663: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:20.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:20 smithi106 ceph-mon[28081]: pgmap v664: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:20.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:20 smithi171 ceph-mon[63127]: pgmap v664: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:20.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:20 smithi129 ceph-mon[62418]: pgmap v664: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:22.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:22 smithi106 ceph-mon[28081]: pgmap v665: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:22.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:22 smithi171 ceph-mon[63127]: pgmap v665: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:22.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:22 smithi129 ceph-mon[62418]: pgmap v665: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:24.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:24 smithi106 ceph-mon[28081]: pgmap v666: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:24.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:24 smithi171 ceph-mon[63127]: pgmap v666: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:24.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:24 smithi129 ceph-mon[62418]: pgmap v666: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:26.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:26 smithi106 ceph-mon[28081]: pgmap v667: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:26.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:26 smithi171 ceph-mon[63127]: pgmap v667: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:26.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:26 smithi129 ceph-mon[62418]: pgmap v667: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:28.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:28 smithi106 ceph-mon[28081]: pgmap v668: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:28.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:28 smithi171 ceph-mon[63127]: pgmap v668: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:28.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:28 smithi129 ceph-mon[62418]: pgmap v668: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:30.577 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:30 smithi106 ceph-mon[28081]: pgmap v669: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:30.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:30 smithi171 ceph-mon[63127]: pgmap v669: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:30.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:30 smithi129 ceph-mon[62418]: pgmap v669: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:32.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:32 smithi106 ceph-mon[28081]: pgmap v670: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:32.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:32 smithi171 ceph-mon[63127]: pgmap v670: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:32.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:32 smithi129 ceph-mon[62418]: pgmap v670: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:34.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:34 smithi106 ceph-mon[28081]: pgmap v671: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:34.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:34 smithi171 ceph-mon[63127]: pgmap v671: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:34.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:34 smithi129 ceph-mon[62418]: pgmap v671: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:36.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:36 smithi106 ceph-mon[28081]: pgmap v672: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:36.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:36 smithi171 ceph-mon[63127]: pgmap v672: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:36.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:36 smithi129 ceph-mon[62418]: pgmap v672: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:38.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:38 smithi106 ceph-mon[28081]: pgmap v673: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:38.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:38 smithi171 ceph-mon[63127]: pgmap v673: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:38.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:38 smithi129 ceph-mon[62418]: pgmap v673: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:40.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:40 smithi171 ceph-mon[63127]: pgmap v674: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:40.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:40 smithi129 ceph-mon[62418]: pgmap v674: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:40.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:40 smithi106 ceph-mon[28081]: pgmap v674: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:42.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:42 smithi171 ceph-mon[63127]: pgmap v675: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:42.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:42 smithi129 ceph-mon[62418]: pgmap v675: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:42.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:42 smithi106 ceph-mon[28081]: pgmap v675: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:44.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:44 smithi171 ceph-mon[63127]: pgmap v676: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:44.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:44 smithi129 ceph-mon[62418]: pgmap v676: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:44.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:44 smithi106 ceph-mon[28081]: pgmap v676: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:46 smithi171 ceph-mon[63127]: pgmap v677: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:46.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:46 smithi129 ceph-mon[62418]: pgmap v677: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:46.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:46 smithi106 ceph-mon[28081]: pgmap v677: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:47.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:00:47.506+0000 7fe054c00640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe030005ea0 con 0x7fe058108fc0 2024-04-28T13:00:47.506 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:00:47.507+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe030005ea0 con 0x7fe058108fc0 2024-04-28T13:00:48.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:48 smithi171 ceph-mon[63127]: pgmap v678: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:48.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:48 smithi129 ceph-mon[62418]: pgmap v678: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:48.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:48 smithi106 ceph-mon[28081]: pgmap v678: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:50.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:50 smithi171 ceph-mon[63127]: pgmap v679: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:50.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:50 smithi129 ceph-mon[62418]: pgmap v679: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:50.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:50 smithi106 ceph-mon[28081]: pgmap v679: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:51.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:51 smithi171 ceph-mon[63127]: pgmap v680: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:51.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:51 smithi129 ceph-mon[62418]: pgmap v680: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:51.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:51 smithi106 ceph-mon[28081]: pgmap v680: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:00:53.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:53 smithi171 ceph-mon[63127]: pgmap v681: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:53.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:53 smithi129 ceph-mon[62418]: pgmap v681: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:53.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:53 smithi106 ceph-mon[28081]: pgmap v681: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:55.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:55 smithi171 ceph-mon[63127]: pgmap v682: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:55.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:55 smithi129 ceph-mon[62418]: pgmap v682: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:55.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:55 smithi106 ceph-mon[28081]: pgmap v682: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:00:57.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:57 smithi171 ceph-mon[63127]: pgmap v683: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:57.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:57 smithi129 ceph-mon[62418]: pgmap v683: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:57.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:57 smithi106 ceph-mon[28081]: pgmap v683: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:00:59.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:00:59 smithi171 ceph-mon[63127]: pgmap v684: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:59.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:00:59 smithi129 ceph-mon[62418]: pgmap v684: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:00:59.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:00:59 smithi106 ceph-mon[28081]: pgmap v684: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:01.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:01 smithi171 ceph-mon[63127]: pgmap v685: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:01.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:01 smithi129 ceph-mon[62418]: pgmap v685: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:01.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:01 smithi106 ceph-mon[28081]: pgmap v685: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:03.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:03 smithi129 ceph-mon[62418]: pgmap v686: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:03.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:03 smithi106 ceph-mon[28081]: pgmap v686: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:03.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:03 smithi171 ceph-mon[63127]: pgmap v686: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:05.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:05 smithi129 ceph-mon[62418]: pgmap v687: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:05.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:05 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:01:05.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:05 smithi106 ceph-mon[28081]: pgmap v687: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:05.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:05 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:01:05.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:05 smithi171 ceph-mon[63127]: pgmap v687: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:05.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:05 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:01:07.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:07 smithi129 ceph-mon[62418]: pgmap v688: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:07.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:01:07.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:01:07.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:01:07.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:07 smithi106 ceph-mon[28081]: pgmap v688: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:07.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:01:07.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:01:07.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:01:07.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:07 smithi171 ceph-mon[63127]: pgmap v688: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:07.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:01:07.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:01:07.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:01:09.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:09 smithi129 ceph-mon[62418]: pgmap v689: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:09.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:09 smithi106 ceph-mon[28081]: pgmap v689: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:09.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:09 smithi171 ceph-mon[63127]: pgmap v689: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:11.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:11 smithi106 ceph-mon[28081]: pgmap v690: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:11.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:11 smithi171 ceph-mon[63127]: pgmap v690: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:12.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:11 smithi129 ceph-mon[62418]: pgmap v690: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:13.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:13 smithi106 ceph-mon[28081]: pgmap v691: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:13.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:13 smithi171 ceph-mon[63127]: pgmap v691: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:14.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:13 smithi129 ceph-mon[62418]: pgmap v691: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:15.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:15 smithi106 ceph-mon[28081]: pgmap v692: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:15.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:15 smithi171 ceph-mon[63127]: pgmap v692: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:16.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:15 smithi129 ceph-mon[62418]: pgmap v692: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:17.801 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:17 smithi171 ceph-mon[63127]: pgmap v693: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:17.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:17 smithi106 ceph-mon[28081]: pgmap v693: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:18.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:17 smithi129 ceph-mon[62418]: pgmap v693: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:19.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:19 smithi106 ceph-mon[28081]: pgmap v694: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:19 smithi171 ceph-mon[63127]: pgmap v694: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:20.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:19 smithi129 ceph-mon[62418]: pgmap v694: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:21.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:21 smithi106 ceph-mon[28081]: pgmap v695: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:21.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:21 smithi171 ceph-mon[63127]: pgmap v695: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:22.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:21 smithi129 ceph-mon[62418]: pgmap v695: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:23.819 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:23 smithi129 ceph-mon[62418]: pgmap v696: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:23.827 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:23 smithi106 ceph-mon[28081]: pgmap v696: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:23.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:23 smithi171 ceph-mon[63127]: pgmap v696: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:25.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:25 smithi106 ceph-mon[28081]: pgmap v697: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:25.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:25 smithi171 ceph-mon[63127]: pgmap v697: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:26.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:25 smithi129 ceph-mon[62418]: pgmap v697: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:27.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:27 smithi106 ceph-mon[28081]: pgmap v698: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:27.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:27 smithi171 ceph-mon[63127]: pgmap v698: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:28.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:27 smithi129 ceph-mon[62418]: pgmap v698: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:29.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:29 smithi106 ceph-mon[28081]: pgmap v699: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:29.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:29 smithi171 ceph-mon[63127]: pgmap v699: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:30.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:29 smithi129 ceph-mon[62418]: pgmap v699: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:31.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:31 smithi106 ceph-mon[28081]: pgmap v700: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:31.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:31 smithi171 ceph-mon[63127]: pgmap v700: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:32.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:31 smithi129 ceph-mon[62418]: pgmap v700: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:33.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:33 smithi106 ceph-mon[28081]: pgmap v701: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:33.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:33 smithi171 ceph-mon[63127]: pgmap v701: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:34.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:33 smithi129 ceph-mon[62418]: pgmap v701: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:35.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:35 smithi171 ceph-mon[63127]: pgmap v702: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:36.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:35 smithi129 ceph-mon[62418]: pgmap v702: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:36.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:35 smithi106 ceph-mon[28081]: pgmap v702: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:37.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:37 smithi171 ceph-mon[63127]: pgmap v703: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:38.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:37 smithi129 ceph-mon[62418]: pgmap v703: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:38.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:37 smithi106 ceph-mon[28081]: pgmap v703: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:39.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:39 smithi171 ceph-mon[63127]: pgmap v704: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:40.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:39 smithi129 ceph-mon[62418]: pgmap v704: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:40.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:39 smithi106 ceph-mon[28081]: pgmap v704: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:41.880 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:41 smithi106 ceph-mon[28081]: pgmap v705: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:41.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:41 smithi171 ceph-mon[63127]: pgmap v705: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:42.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:41 smithi129 ceph-mon[62418]: pgmap v705: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:43.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:43 smithi171 ceph-mon[63127]: pgmap v706: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:44.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:43 smithi129 ceph-mon[62418]: pgmap v706: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:44.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:43 smithi106 ceph-mon[28081]: pgmap v706: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:45.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:45 smithi171 ceph-mon[63127]: pgmap v707: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:46.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:45 smithi129 ceph-mon[62418]: pgmap v707: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:46.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:45 smithi106 ceph-mon[28081]: pgmap v707: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:47.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:47 smithi171 ceph-mon[63127]: pgmap v708: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:48.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:47 smithi129 ceph-mon[62418]: pgmap v708: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:48.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:47 smithi106 ceph-mon[28081]: pgmap v708: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:49.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:49 smithi171 ceph-mon[63127]: pgmap v709: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:50.016 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:49 smithi129 ceph-mon[62418]: pgmap v709: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:50.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:49 smithi106 ceph-mon[28081]: pgmap v709: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:01:52.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:52 smithi171 ceph-mon[63127]: pgmap v710: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:52.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:52 smithi129 ceph-mon[62418]: pgmap v710: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:52.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:52 smithi106 ceph-mon[28081]: pgmap v710: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:01:54.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:54 smithi129 ceph-mon[62418]: pgmap v711: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:54.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:54 smithi106 ceph-mon[28081]: pgmap v711: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:54.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:54 smithi171 ceph-mon[63127]: pgmap v711: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:56.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:56 smithi129 ceph-mon[62418]: pgmap v712: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:56.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:56 smithi106 ceph-mon[28081]: pgmap v712: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:56.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:56 smithi171 ceph-mon[63127]: pgmap v712: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:01:58.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:01:58 smithi129 ceph-mon[62418]: pgmap v713: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:58.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:01:58 smithi106 ceph-mon[28081]: pgmap v713: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:01:58.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:01:58 smithi171 ceph-mon[63127]: pgmap v713: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:00.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:00 smithi106 ceph-mon[28081]: pgmap v714: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:00.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:00 smithi171 ceph-mon[63127]: pgmap v714: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:00.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:00 smithi129 ceph-mon[62418]: pgmap v714: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:02.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:02 smithi106 ceph-mon[28081]: pgmap v715: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:02.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:02 smithi171 ceph-mon[63127]: pgmap v715: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:02.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:02 smithi129 ceph-mon[62418]: pgmap v715: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:04.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:04 smithi106 ceph-mon[28081]: pgmap v716: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:04.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:04 smithi171 ceph-mon[63127]: pgmap v716: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:04.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:04 smithi129 ceph-mon[62418]: pgmap v716: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:06.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:06 smithi106 ceph-mon[28081]: pgmap v717: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:06.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:06 smithi171 ceph-mon[63127]: pgmap v717: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:06.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:06 smithi129 ceph-mon[62418]: pgmap v717: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:07.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:07 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:02:07.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:07 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:02:07.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:07 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:02:08.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:08 smithi106 ceph-mon[28081]: pgmap v718: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:08.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:08 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:08 smithi171 ceph-mon[63127]: pgmap v718: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:08.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.632 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:08 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.666 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:08 smithi129 ceph-mon[62418]: pgmap v718: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:08.666 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.666 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.666 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.666 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.666 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:08.667 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:08 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:10 smithi106 ceph-mon[28081]: pgmap v719: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:10.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:10 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:10 smithi171 ceph-mon[63127]: pgmap v719: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:10.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:10 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:10 smithi129 ceph-mon[62418]: pgmap v719: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:10.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:10.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:10 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:12.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:11 smithi129 ceph-mon[62418]: pgmap v720: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:12.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:12.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:11 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:12.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:11 smithi106 ceph-mon[28081]: pgmap v720: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:12.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:12.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:11 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:12.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:11 smithi171 ceph-mon[63127]: pgmap v720: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:12.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:12.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:11 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:13.077 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:02:13.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:02:13.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:12 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:13.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:02:13.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:02:13.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:12 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:13.266 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:02:13.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:02:13.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:12 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:02:14.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:14 smithi171 ceph-mon[63127]: pgmap v721: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:14.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:14 smithi129 ceph-mon[62418]: pgmap v721: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:14.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:14 smithi106 ceph-mon[28081]: pgmap v721: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:16.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:16 smithi171 ceph-mon[63127]: pgmap v722: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:16.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:16 smithi129 ceph-mon[62418]: pgmap v722: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:16.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:16 smithi106 ceph-mon[28081]: pgmap v722: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:18.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:18 smithi171 ceph-mon[63127]: pgmap v723: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:18.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:18 smithi129 ceph-mon[62418]: pgmap v723: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:18.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:18 smithi106 ceph-mon[28081]: pgmap v723: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:20 smithi171 ceph-mon[63127]: pgmap v724: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:20.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:20 smithi129 ceph-mon[62418]: pgmap v724: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:20.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:20 smithi106 ceph-mon[28081]: pgmap v724: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:22.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:22 smithi171 ceph-mon[63127]: pgmap v725: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:22.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:22 smithi129 ceph-mon[62418]: pgmap v725: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:22.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:22 smithi106 ceph-mon[28081]: pgmap v725: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:24.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:24 smithi171 ceph-mon[63127]: pgmap v726: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:24.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:24 smithi129 ceph-mon[62418]: pgmap v726: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:24.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:24 smithi106 ceph-mon[28081]: pgmap v726: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:26.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:26 smithi171 ceph-mon[63127]: pgmap v727: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:26.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:26 smithi129 ceph-mon[62418]: pgmap v727: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:26.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:26 smithi106 ceph-mon[28081]: pgmap v727: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:28.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:28 smithi171 ceph-mon[63127]: pgmap v728: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:28.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:28 smithi129 ceph-mon[62418]: pgmap v728: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:28.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:28 smithi106 ceph-mon[28081]: pgmap v728: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:30 smithi171 ceph-mon[63127]: pgmap v729: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:30.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:30 smithi129 ceph-mon[62418]: pgmap v729: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:30.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:30 smithi106 ceph-mon[28081]: pgmap v729: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:32.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:32 smithi171 ceph-mon[63127]: pgmap v730: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:32.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:32 smithi129 ceph-mon[62418]: pgmap v730: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:32.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:32 smithi106 ceph-mon[28081]: pgmap v730: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:34.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:34 smithi171 ceph-mon[63127]: pgmap v731: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:34.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:34 smithi129 ceph-mon[62418]: pgmap v731: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:34.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:34 smithi106 ceph-mon[28081]: pgmap v731: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:36.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:36 smithi171 ceph-mon[63127]: pgmap v732: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:36.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:36 smithi129 ceph-mon[62418]: pgmap v732: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:36.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:36 smithi106 ceph-mon[28081]: pgmap v732: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:38.500 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:38 smithi171 ceph-mon[63127]: pgmap v733: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:38.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:38 smithi129 ceph-mon[62418]: pgmap v733: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:38.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:38 smithi106 ceph-mon[28081]: pgmap v733: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:40.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:40 smithi171 ceph-mon[63127]: pgmap v734: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:40.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:40 smithi129 ceph-mon[62418]: pgmap v734: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:40.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:40 smithi106 ceph-mon[28081]: pgmap v734: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:42.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:42 smithi171 ceph-mon[63127]: pgmap v735: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:42.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:42 smithi129 ceph-mon[62418]: pgmap v735: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:42.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:42 smithi106 ceph-mon[28081]: pgmap v735: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:44.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:44 smithi171 ceph-mon[63127]: pgmap v736: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:44.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:44 smithi129 ceph-mon[62418]: pgmap v736: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:44.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:44 smithi106 ceph-mon[28081]: pgmap v736: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:46.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:46 smithi171 ceph-mon[63127]: pgmap v737: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:46.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:46 smithi129 ceph-mon[62418]: pgmap v737: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:46.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:46 smithi106 ceph-mon[28081]: pgmap v737: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:48.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:48 smithi171 ceph-mon[63127]: pgmap v738: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:48.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:48 smithi129 ceph-mon[62418]: pgmap v738: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:48.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:48 smithi106 ceph-mon[28081]: pgmap v738: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:50.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:50 smithi171 ceph-mon[63127]: pgmap v739: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:50.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:50 smithi129 ceph-mon[62418]: pgmap v739: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:50.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:50 smithi106 ceph-mon[28081]: pgmap v739: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-28T13:02:52.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:52 smithi171 ceph-mon[63127]: pgmap v740: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:52.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:52 smithi129 ceph-mon[62418]: pgmap v740: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:52.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:52 smithi106 ceph-mon[28081]: pgmap v740: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:02:54.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:54 smithi171 ceph-mon[63127]: pgmap v741: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:54.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:54 smithi129 ceph-mon[62418]: pgmap v741: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:54.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:54 smithi106 ceph-mon[28081]: pgmap v741: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:56.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:56 smithi171 ceph-mon[63127]: pgmap v742: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:56.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:56 smithi129 ceph-mon[62418]: pgmap v742: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:56.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:56 smithi106 ceph-mon[28081]: pgmap v742: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:02:58.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:02:58 smithi171 ceph-mon[63127]: pgmap v743: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:58.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:02:58 smithi129 ceph-mon[62418]: pgmap v743: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:02:58.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:02:58 smithi106 ceph-mon[28081]: pgmap v743: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:00.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:00 smithi171 ceph-mon[63127]: pgmap v744: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:00.516 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:00 smithi129 ceph-mon[62418]: pgmap v744: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:00.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:00 smithi106 ceph-mon[28081]: pgmap v744: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:02.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:02 smithi129 ceph-mon[62418]: pgmap v745: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:03:02.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:02 smithi106 ceph-mon[28081]: pgmap v745: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:03:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:02 smithi171 ceph-mon[63127]: pgmap v745: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-28T13:03:04.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:04 smithi129 ceph-mon[62418]: pgmap v746: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:04.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:04 smithi106 ceph-mon[28081]: pgmap v746: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:04.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:04 smithi171 ceph-mon[63127]: pgmap v746: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:06.517 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:06 smithi129 ceph-mon[62418]: pgmap v747: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:06.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:06 smithi106 ceph-mon[28081]: pgmap v747: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:06.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:06 smithi171 ceph-mon[63127]: pgmap v747: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:08.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:08 smithi106 ceph-mon[28081]: pgmap v748: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:08.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:08 smithi171 ceph-mon[63127]: pgmap v748: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:08.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:08 smithi129 ceph-mon[62418]: pgmap v748: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:10.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:10 smithi106 ceph-mon[28081]: pgmap v749: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:10.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:10 smithi171 ceph-mon[63127]: pgmap v749: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:10.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:10 smithi129 ceph-mon[62418]: pgmap v749: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:12.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:12 smithi106 ceph-mon[28081]: pgmap v750: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:12.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:12 smithi171 ceph-mon[63127]: pgmap v750: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:12.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:12 smithi129 ceph-mon[62418]: pgmap v750: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:13.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:13 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:03:13.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:13 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:03:13.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:13 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:03:14.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:14 smithi106 ceph-mon[28081]: pgmap v751: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:14.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:03:14.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:03:14.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:03:14.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:14 smithi171 ceph-mon[63127]: pgmap v751: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:14.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:03:14.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:03:14.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:03:14.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:14 smithi129 ceph-mon[62418]: pgmap v751: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:14.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:03:14.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:03:14.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:03:16.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:16 smithi106 ceph-mon[28081]: pgmap v752: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:03:16.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:16 smithi171 ceph-mon[63127]: pgmap v752: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:03:16.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:16 smithi129 ceph-mon[62418]: pgmap v752: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:03:18.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:18 smithi106 ceph-mon[28081]: pgmap v753: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:03:18.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:18 smithi171 ceph-mon[63127]: pgmap v753: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:03:18.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:18 smithi129 ceph-mon[62418]: pgmap v753: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:03:20.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:20 smithi106 ceph-mon[28081]: pgmap v754: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:03:20.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:20 smithi171 ceph-mon[63127]: pgmap v754: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:03:20.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:20 smithi129 ceph-mon[62418]: pgmap v754: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:03:22.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:22 smithi106 ceph-mon[28081]: pgmap v755: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T13:03:22.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:22 smithi171 ceph-mon[63127]: pgmap v755: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T13:03:22.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:22 smithi129 ceph-mon[62418]: pgmap v755: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T13:03:24.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:24 smithi171 ceph-mon[63127]: pgmap v756: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:03:24.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:24 smithi129 ceph-mon[62418]: pgmap v756: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:03:24.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:24 smithi106 ceph-mon[28081]: pgmap v756: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:03:26.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:26 smithi171 ceph-mon[63127]: pgmap v757: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:03:26.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:26 smithi129 ceph-mon[62418]: pgmap v757: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:03:26.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:26 smithi106 ceph-mon[28081]: pgmap v757: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:03:28.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:28 smithi171 ceph-mon[63127]: pgmap v758: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:28.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:28 smithi129 ceph-mon[62418]: pgmap v758: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:28.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:28 smithi106 ceph-mon[28081]: pgmap v758: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:30.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:30 smithi171 ceph-mon[63127]: pgmap v759: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:30.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:30 smithi129 ceph-mon[62418]: pgmap v759: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:30.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:30 smithi106 ceph-mon[28081]: pgmap v759: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:32.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:32 smithi171 ceph-mon[63127]: pgmap v760: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:32.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:32 smithi129 ceph-mon[62418]: pgmap v760: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:32.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:32 smithi106 ceph-mon[28081]: pgmap v760: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:34.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:34 smithi171 ceph-mon[63127]: pgmap v761: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:34.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:34 smithi129 ceph-mon[62418]: pgmap v761: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:34.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:34 smithi106 ceph-mon[28081]: pgmap v761: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:36.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:36 smithi171 ceph-mon[63127]: pgmap v762: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:36.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:36 smithi129 ceph-mon[62418]: pgmap v762: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:36.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:36 smithi106 ceph-mon[28081]: pgmap v762: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:38.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:38 smithi171 ceph-mon[63127]: pgmap v763: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:38.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:38 smithi129 ceph-mon[62418]: pgmap v763: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:38.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:38 smithi106 ceph-mon[28081]: pgmap v763: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:40.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:40 smithi171 ceph-mon[63127]: pgmap v764: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:40.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:40 smithi129 ceph-mon[62418]: pgmap v764: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:40.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:40 smithi106 ceph-mon[28081]: pgmap v764: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:42.631 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:42 smithi106 ceph-mon[28081]: pgmap v765: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:42.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:42 smithi171 ceph-mon[63127]: pgmap v765: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:42.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:42 smithi129 ceph-mon[62418]: pgmap v765: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:44.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:44 smithi171 ceph-mon[63127]: pgmap v766: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:44.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:44 smithi129 ceph-mon[62418]: pgmap v766: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:44.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:44 smithi106 ceph-mon[28081]: pgmap v766: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:46.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:46 smithi171 ceph-mon[63127]: pgmap v767: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:46.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:46 smithi129 ceph-mon[62418]: pgmap v767: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:46.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:46 smithi106 ceph-mon[28081]: pgmap v767: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:03:47.536+0000 7fe054c00640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe0300060c0 con 0x7fe058108fc0 2024-04-28T13:03:47.536 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:03:47.537+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe0300060c0 con 0x7fe058108fc0 2024-04-28T13:03:48.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:48 smithi171 ceph-mon[63127]: pgmap v768: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:48.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:48 smithi129 ceph-mon[62418]: pgmap v768: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:48.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:48 smithi106 ceph-mon[28081]: pgmap v768: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:50.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:50 smithi171 ceph-mon[63127]: pgmap v769: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:50.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:50 smithi129 ceph-mon[62418]: pgmap v769: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:50.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:50 smithi106 ceph-mon[28081]: pgmap v769: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:52.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:52 smithi106 ceph-mon[28081]: pgmap v770: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:52.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:52 smithi171 ceph-mon[63127]: pgmap v770: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:52.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:52 smithi129 ceph-mon[62418]: pgmap v770: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:03:54.569 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:54 smithi129 ceph-mon[62418]: pgmap v771: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:54.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:54 smithi106 ceph-mon[28081]: pgmap v771: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:54.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:54 smithi171 ceph-mon[63127]: pgmap v771: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:56.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:56 smithi106 ceph-mon[28081]: pgmap v772: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:56.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:56 smithi171 ceph-mon[63127]: pgmap v772: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:56.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:56 smithi129 ceph-mon[62418]: pgmap v772: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:03:58.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:03:58 smithi106 ceph-mon[28081]: pgmap v773: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:58.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:03:58 smithi171 ceph-mon[63127]: pgmap v773: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:03:58.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:03:58 smithi129 ceph-mon[62418]: pgmap v773: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:00.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:00 smithi106 ceph-mon[28081]: pgmap v774: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:00.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:00 smithi171 ceph-mon[63127]: pgmap v774: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:00.766 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:00 smithi129 ceph-mon[62418]: pgmap v774: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:02.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:02 smithi106 ceph-mon[28081]: pgmap v775: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:02.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:02 smithi171 ceph-mon[63127]: pgmap v775: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:02.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:02 smithi129 ceph-mon[62418]: pgmap v775: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:04.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:04 smithi106 ceph-mon[28081]: pgmap v776: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:04.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:04 smithi171 ceph-mon[63127]: pgmap v776: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:04.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:04 smithi129 ceph-mon[62418]: pgmap v776: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:06.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:06 smithi171 ceph-mon[63127]: pgmap v777: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:06.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:06 smithi129 ceph-mon[62418]: pgmap v777: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:06.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:06 smithi106 ceph-mon[28081]: pgmap v777: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:08.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:08 smithi171 ceph-mon[63127]: pgmap v778: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:08.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:08 smithi129 ceph-mon[62418]: pgmap v778: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:08.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:08 smithi106 ceph-mon[28081]: pgmap v778: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:10.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:10 smithi171 ceph-mon[63127]: pgmap v779: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:10.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:10 smithi129 ceph-mon[62418]: pgmap v779: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:10.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:10 smithi106 ceph-mon[28081]: pgmap v779: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:12.631 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:12 smithi106 ceph-mon[28081]: pgmap v780: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:12.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:12 smithi171 ceph-mon[63127]: pgmap v780: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:12.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:12 smithi129 ceph-mon[62418]: pgmap v780: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:14.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:14 smithi171 ceph-mon[63127]: pgmap v781: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:14.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:14 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:04:14.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:14 smithi129 ceph-mon[62418]: pgmap v781: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:14.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:14 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:04:14.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:14 smithi106 ceph-mon[28081]: pgmap v781: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:14.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:14 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:04:15.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:04:15.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:04:15.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:04:15.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:04:15.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:04:15.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:04:15.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:04:15.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:04:15.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:04:16.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:16 smithi171 ceph-mon[63127]: pgmap v782: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:16.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:16 smithi129 ceph-mon[62418]: pgmap v782: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:16.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:16 smithi106 ceph-mon[28081]: pgmap v782: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:18.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:18 smithi171 ceph-mon[63127]: pgmap v783: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:18.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:18 smithi129 ceph-mon[62418]: pgmap v783: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:18.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:18 smithi106 ceph-mon[28081]: pgmap v783: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:20.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:20 smithi171 ceph-mon[63127]: pgmap v784: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:20.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:20 smithi129 ceph-mon[62418]: pgmap v784: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:20.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:20 smithi106 ceph-mon[28081]: pgmap v784: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:22.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:22 smithi171 ceph-mon[63127]: pgmap v785: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:22.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:22 smithi129 ceph-mon[62418]: pgmap v785: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:22.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:22 smithi106 ceph-mon[28081]: pgmap v785: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:24.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:24 smithi171 ceph-mon[63127]: pgmap v786: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:24.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:24 smithi129 ceph-mon[62418]: pgmap v786: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:24.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:24 smithi106 ceph-mon[28081]: pgmap v786: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:26.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:26 smithi171 ceph-mon[63127]: pgmap v787: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:26.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:26 smithi129 ceph-mon[62418]: pgmap v787: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:26.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:26 smithi106 ceph-mon[28081]: pgmap v787: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:28.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:28 smithi171 ceph-mon[63127]: pgmap v788: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:28.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:28 smithi129 ceph-mon[62418]: pgmap v788: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:28.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:28 smithi106 ceph-mon[28081]: pgmap v788: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:30.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:30 smithi171 ceph-mon[63127]: pgmap v789: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:30.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:30 smithi129 ceph-mon[62418]: pgmap v789: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:30.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:30 smithi106 ceph-mon[28081]: pgmap v789: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:32.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:32 smithi171 ceph-mon[63127]: pgmap v790: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:32.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:32 smithi129 ceph-mon[62418]: pgmap v790: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:32.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:32 smithi106 ceph-mon[28081]: pgmap v790: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:34.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:34 smithi171 ceph-mon[63127]: pgmap v791: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:34.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:34 smithi129 ceph-mon[62418]: pgmap v791: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:34.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:34 smithi106 ceph-mon[28081]: pgmap v791: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:35.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:35 smithi171 ceph-mon[63127]: pgmap v792: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:35.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:35 smithi129 ceph-mon[62418]: pgmap v792: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:35.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:35 smithi106 ceph-mon[28081]: pgmap v792: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:37.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:37 smithi171 ceph-mon[63127]: pgmap v793: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:38.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:37 smithi129 ceph-mon[62418]: pgmap v793: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:38.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:37 smithi106 ceph-mon[28081]: pgmap v793: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:39.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:39 smithi171 ceph-mon[63127]: pgmap v794: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:40.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:39 smithi129 ceph-mon[62418]: pgmap v794: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:40.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:39 smithi106 ceph-mon[28081]: pgmap v794: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:41.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:41 smithi171 ceph-mon[63127]: pgmap v795: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:42.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:41 smithi129 ceph-mon[62418]: pgmap v795: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:42.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:41 smithi106 ceph-mon[28081]: pgmap v795: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:04:43.881 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:43 smithi106 ceph-mon[28081]: pgmap v796: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T13:04:43.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:43 smithi171 ceph-mon[63127]: pgmap v796: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T13:04:44.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:43 smithi129 ceph-mon[62418]: pgmap v796: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T13:04:45.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:45 smithi171 ceph-mon[63127]: pgmap v797: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-28T13:04:46.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:45 smithi129 ceph-mon[62418]: pgmap v797: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-28T13:04:46.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:45 smithi106 ceph-mon[28081]: pgmap v797: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-28T13:04:47.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:47 smithi171 ceph-mon[63127]: pgmap v798: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T13:04:48.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:47 smithi129 ceph-mon[62418]: pgmap v798: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T13:04:48.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:47 smithi106 ceph-mon[28081]: pgmap v798: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T13:04:49.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:49 smithi171 ceph-mon[63127]: pgmap v799: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-28T13:04:50.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:49 smithi129 ceph-mon[62418]: pgmap v799: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-28T13:04:50.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:49 smithi106 ceph-mon[28081]: pgmap v799: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-04-28T13:04:51.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:51 smithi171 ceph-mon[63127]: pgmap v800: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-04-28T13:04:52.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:51 smithi129 ceph-mon[62418]: pgmap v800: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-04-28T13:04:52.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:51 smithi106 ceph-mon[28081]: pgmap v800: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-04-28T13:04:53.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:53 smithi171 ceph-mon[63127]: pgmap v801: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T13:04:54.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:53 smithi129 ceph-mon[62418]: pgmap v801: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T13:04:54.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:53 smithi106 ceph-mon[28081]: pgmap v801: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-04-28T13:04:55.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:55 smithi171 ceph-mon[63127]: pgmap v802: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:56.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:55 smithi129 ceph-mon[62418]: pgmap v802: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:56.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:55 smithi106 ceph-mon[28081]: pgmap v802: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:04:57.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:57 smithi171 ceph-mon[63127]: pgmap v803: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:58.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:57 smithi129 ceph-mon[62418]: pgmap v803: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:58.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:57 smithi106 ceph-mon[28081]: pgmap v803: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:04:59.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:04:59 smithi171 ceph-mon[63127]: pgmap v804: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:00.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:04:59 smithi129 ceph-mon[62418]: pgmap v804: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:00.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:04:59 smithi106 ceph-mon[28081]: pgmap v804: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:01.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:01 smithi171 ceph-mon[63127]: pgmap v805: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:02.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:01 smithi129 ceph-mon[62418]: pgmap v805: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:02.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:01 smithi106 ceph-mon[28081]: pgmap v805: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:03.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:03 smithi171 ceph-mon[63127]: pgmap v806: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:04.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:03 smithi129 ceph-mon[62418]: pgmap v806: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:04.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:03 smithi106 ceph-mon[28081]: pgmap v806: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:05.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:05 smithi171 ceph-mon[63127]: pgmap v807: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:06.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:05 smithi129 ceph-mon[62418]: pgmap v807: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:06.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:05 smithi106 ceph-mon[28081]: pgmap v807: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:07.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:07 smithi171 ceph-mon[63127]: pgmap v808: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:08.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:07 smithi129 ceph-mon[62418]: pgmap v808: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:08.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:07 smithi106 ceph-mon[28081]: pgmap v808: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:09.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:09 smithi171 ceph-mon[63127]: pgmap v809: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:10.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:09 smithi129 ceph-mon[62418]: pgmap v809: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:10.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:09 smithi106 ceph-mon[28081]: pgmap v809: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:11.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:11 smithi171 ceph-mon[63127]: pgmap v810: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:12.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:11 smithi129 ceph-mon[62418]: pgmap v810: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:12.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:11 smithi106 ceph-mon[28081]: pgmap v810: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:13.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:13 smithi171 ceph-mon[63127]: pgmap v811: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:14.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:13 smithi129 ceph-mon[62418]: pgmap v811: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:14.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:13 smithi106 ceph-mon[28081]: pgmap v811: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:15.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:15 smithi171 ceph-mon[63127]: pgmap v812: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:15.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:15 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:05:16.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:15 smithi129 ceph-mon[62418]: pgmap v812: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:16.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:15 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:05:16.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:15 smithi106 ceph-mon[28081]: pgmap v812: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:16.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:15 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:05:17.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:17 smithi171 ceph-mon[63127]: pgmap v813: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:17.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:05:17.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:05:17.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:05:18.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:17 smithi129 ceph-mon[62418]: pgmap v813: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:18.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:05:18.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:05:18.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:05:18.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:17 smithi106 ceph-mon[28081]: pgmap v813: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:18.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:05:18.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:05:18.079 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:05:19.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:19 smithi171 ceph-mon[63127]: pgmap v814: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:20.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:19 smithi129 ceph-mon[62418]: pgmap v814: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:20.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:19 smithi106 ceph-mon[28081]: pgmap v814: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:21.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:21 smithi171 ceph-mon[63127]: pgmap v815: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:22.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:21 smithi129 ceph-mon[62418]: pgmap v815: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:22.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:21 smithi106 ceph-mon[28081]: pgmap v815: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:23.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:23 smithi171 ceph-mon[63127]: pgmap v816: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:24.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:23 smithi129 ceph-mon[62418]: pgmap v816: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:24.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:23 smithi106 ceph-mon[28081]: pgmap v816: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:25.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:25 smithi171 ceph-mon[63127]: pgmap v817: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:26.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:25 smithi129 ceph-mon[62418]: pgmap v817: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:26.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:25 smithi106 ceph-mon[28081]: pgmap v817: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:27.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:27 smithi171 ceph-mon[63127]: pgmap v818: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:28.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:27 smithi129 ceph-mon[62418]: pgmap v818: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:28.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:27 smithi106 ceph-mon[28081]: pgmap v818: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:29.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:29 smithi171 ceph-mon[63127]: pgmap v819: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:30.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:29 smithi129 ceph-mon[62418]: pgmap v819: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:30.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:29 smithi106 ceph-mon[28081]: pgmap v819: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:31.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:31 smithi171 ceph-mon[63127]: pgmap v820: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:32.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:31 smithi129 ceph-mon[62418]: pgmap v820: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:32.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:31 smithi106 ceph-mon[28081]: pgmap v820: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:33.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:33 smithi171 ceph-mon[63127]: pgmap v821: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:34.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:33 smithi129 ceph-mon[62418]: pgmap v821: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:34.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:33 smithi106 ceph-mon[28081]: pgmap v821: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:35.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:35 smithi171 ceph-mon[63127]: pgmap v822: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:36.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:35 smithi129 ceph-mon[62418]: pgmap v822: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:36.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:35 smithi106 ceph-mon[28081]: pgmap v822: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:38.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:37 smithi129 ceph-mon[62418]: pgmap v823: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:38.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:37 smithi106 ceph-mon[28081]: pgmap v823: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:38.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:37 smithi171 ceph-mon[63127]: pgmap v823: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:40.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:39 smithi129 ceph-mon[62418]: pgmap v824: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:40.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:39 smithi106 ceph-mon[28081]: pgmap v824: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:40.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:39 smithi171 ceph-mon[63127]: pgmap v824: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:42.017 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:41 smithi129 ceph-mon[62418]: pgmap v825: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:42.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:41 smithi106 ceph-mon[28081]: pgmap v825: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:42.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:41 smithi171 ceph-mon[63127]: pgmap v825: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:05:44.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:43 smithi106 ceph-mon[28081]: pgmap v826: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:44.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:43 smithi171 ceph-mon[63127]: pgmap v826: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:44.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:43 smithi129 ceph-mon[62418]: pgmap v826: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:05:46.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:45 smithi106 ceph-mon[28081]: pgmap v827: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:46.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:45 smithi171 ceph-mon[63127]: pgmap v827: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:46.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:45 smithi129 ceph-mon[62418]: pgmap v827: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:05:48.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:47 smithi106 ceph-mon[28081]: pgmap v828: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:05:48.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:47 smithi171 ceph-mon[63127]: pgmap v828: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:05:48.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:47 smithi129 ceph-mon[62418]: pgmap v828: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:05:50.051 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:49 smithi171 ceph-mon[63127]: pgmap v829: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:05:50.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:49 smithi106 ceph-mon[28081]: pgmap v829: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:05:50.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:49 smithi129 ceph-mon[62418]: pgmap v829: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:05:52.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:51 smithi106 ceph-mon[28081]: pgmap v830: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T13:05:52.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:51 smithi171 ceph-mon[63127]: pgmap v830: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T13:05:52.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:51 smithi129 ceph-mon[62418]: pgmap v830: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T13:05:54.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:53 smithi106 ceph-mon[28081]: pgmap v831: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:05:54.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:53 smithi171 ceph-mon[63127]: pgmap v831: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:05:54.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:53 smithi129 ceph-mon[62418]: pgmap v831: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:05:56.069 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:55 smithi129 ceph-mon[62418]: pgmap v832: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:05:56.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:55 smithi106 ceph-mon[28081]: pgmap v832: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:05:56.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:55 smithi171 ceph-mon[63127]: pgmap v832: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:05:58.081 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:57 smithi106 ceph-mon[28081]: pgmap v833: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:05:58.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:57 smithi171 ceph-mon[63127]: pgmap v833: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:05:58.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:57 smithi129 ceph-mon[62418]: pgmap v833: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:06:00.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:05:59 smithi106 ceph-mon[28081]: pgmap v834: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:00.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:05:59 smithi171 ceph-mon[63127]: pgmap v834: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:00.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:05:59 smithi129 ceph-mon[62418]: pgmap v834: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:02.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:01 smithi106 ceph-mon[28081]: pgmap v835: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:02.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:01 smithi171 ceph-mon[63127]: pgmap v835: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:02.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:01 smithi129 ceph-mon[62418]: pgmap v835: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:04.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:03 smithi106 ceph-mon[28081]: pgmap v836: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:04.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:03 smithi171 ceph-mon[63127]: pgmap v836: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:04.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:03 smithi129 ceph-mon[62418]: pgmap v836: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:06.078 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:05 smithi106 ceph-mon[28081]: pgmap v837: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:06:06.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:05 smithi171 ceph-mon[63127]: pgmap v837: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:06:06.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:05 smithi129 ceph-mon[62418]: pgmap v837: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:06:08.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:07 smithi171 ceph-mon[63127]: pgmap v838: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:06:08.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:07 smithi129 ceph-mon[62418]: pgmap v838: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:06:08.328 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:07 smithi106 ceph-mon[28081]: pgmap v838: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:06:10.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:09 smithi171 ceph-mon[63127]: pgmap v839: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:06:10.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:09 smithi129 ceph-mon[62418]: pgmap v839: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:06:10.328 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:09 smithi106 ceph-mon[28081]: pgmap v839: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:06:12.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:11 smithi171 ceph-mon[63127]: pgmap v840: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T13:06:12.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:11 smithi129 ceph-mon[62418]: pgmap v840: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T13:06:12.328 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:11 smithi106 ceph-mon[28081]: pgmap v840: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-04-28T13:06:14.131 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:13 smithi106 ceph-mon[28081]: pgmap v841: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:06:14.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:13 smithi171 ceph-mon[63127]: pgmap v841: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:06:14.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:13 smithi129 ceph-mon[62418]: pgmap v841: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-28T13:06:16.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:15 smithi171 ceph-mon[63127]: pgmap v842: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:06:16.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:15 smithi129 ceph-mon[62418]: pgmap v842: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:06:16.328 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:15 smithi106 ceph-mon[28081]: pgmap v842: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-28T13:06:18.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:17 smithi171 ceph-mon[63127]: pgmap v843: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:18.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:17 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:06:18.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:17 smithi129 ceph-mon[62418]: pgmap v843: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:18.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:17 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:06:18.328 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:17 smithi106 ceph-mon[28081]: pgmap v843: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:18.328 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:17 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-28T13:06:19.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:18 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:06:19.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:18 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:06:19.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:18 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:06:19.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:18 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:06:19.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:18 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:06:19.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:18 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:06:19.328 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-28T13:06:19.329 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-28T13:06:19.329 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:18 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' 2024-04-28T13:06:20.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:20 smithi106 ceph-mon[28081]: pgmap v844: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:20.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:20 smithi171 ceph-mon[63127]: pgmap v844: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:20.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:20 smithi129 ceph-mon[62418]: pgmap v844: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:22.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:22 smithi106 ceph-mon[28081]: pgmap v845: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:22.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:22 smithi171 ceph-mon[63127]: pgmap v845: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:22.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:22 smithi129 ceph-mon[62418]: pgmap v845: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:24.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:24 smithi106 ceph-mon[28081]: pgmap v846: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:24.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:24 smithi171 ceph-mon[63127]: pgmap v846: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:24.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:24 smithi129 ceph-mon[62418]: pgmap v846: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:26.569 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:26 smithi129 ceph-mon[62418]: pgmap v847: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:26.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:26 smithi106 ceph-mon[28081]: pgmap v847: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:26.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:26 smithi171 ceph-mon[63127]: pgmap v847: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:28.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:28 smithi106 ceph-mon[28081]: pgmap v848: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:28.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:28 smithi171 ceph-mon[63127]: pgmap v848: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:28.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:28 smithi129 ceph-mon[62418]: pgmap v848: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:30.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:30 smithi106 ceph-mon[28081]: pgmap v849: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:30.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:30 smithi171 ceph-mon[63127]: pgmap v849: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:30.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:30 smithi129 ceph-mon[62418]: pgmap v849: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:32.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:32 smithi106 ceph-mon[28081]: pgmap v850: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:32.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:32 smithi171 ceph-mon[63127]: pgmap v850: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:32.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:32 smithi129 ceph-mon[62418]: pgmap v850: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:34.578 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:34 smithi106 ceph-mon[28081]: pgmap v851: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:34.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:34 smithi171 ceph-mon[63127]: pgmap v851: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:34.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:34 smithi129 ceph-mon[62418]: pgmap v851: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:36.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:36 smithi106 ceph-mon[28081]: pgmap v852: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:36.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:36 smithi171 ceph-mon[63127]: pgmap v852: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:36.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:36 smithi129 ceph-mon[62418]: pgmap v852: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:38.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:38 smithi171 ceph-mon[63127]: pgmap v853: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:38.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:38 smithi129 ceph-mon[62418]: pgmap v853: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:38.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:38 smithi106 ceph-mon[28081]: pgmap v853: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:40.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:40 smithi171 ceph-mon[63127]: pgmap v854: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:40.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:40 smithi129 ceph-mon[62418]: pgmap v854: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:40.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:40 smithi106 ceph-mon[28081]: pgmap v854: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:42.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:42 smithi171 ceph-mon[63127]: pgmap v855: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:42.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:42 smithi129 ceph-mon[62418]: pgmap v855: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:42.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:42 smithi106 ceph-mon[28081]: pgmap v855: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:44.631 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:44 smithi106 ceph-mon[28081]: pgmap v856: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:44.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:44 smithi171 ceph-mon[63127]: pgmap v856: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:44.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:44 smithi129 ceph-mon[62418]: pgmap v856: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:46.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:46 smithi171 ceph-mon[63127]: pgmap v857: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:46.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:46 smithi129 ceph-mon[62418]: pgmap v857: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:46.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:46 smithi106 ceph-mon[28081]: pgmap v857: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:47.453 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.454+0000 7fe04fe00640 1 -- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0a3d70 msgr2=0x7fe03c0a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900029267) for more than 900000000 us, fault. 2024-04-28T13:06:47.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.454+0000 7fe04fe00640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0a3d70 0x7fe03c0a6230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe040009230 tx=0x7fe0400058d0 comp rx=0 tx=0).stop 2024-04-28T13:06:47.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.454+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0a3d70 msgr2=0x7fe03c0a6230 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-28T13:06:47.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.454+0000 7fe056000640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0a3d70 0x7fe03c0a6230 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:47.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.454+0000 7fe056000640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0af540 0x7fe03c0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:47.454 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.454+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fe048005620 con 0x7fe03c0af540 2024-04-28T13:06:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.454+0000 7fe04fe00640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0af540 0x7fe03c0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:47.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.456+0000 7fe04fe00640 1 --2- 172.21.15.106:0/3510684457 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fe03c0af540 0x7fe03c0b1930 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe0400096c0 tx=0x7fe040041000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:47.565 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.566+0000 7fe054c00640 1 -- 172.21.15.106:0/3510684457 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7fe030006330 con 0x7fe058108fc0 2024-04-28T13:06:47.566 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:47.566+0000 7fe056000640 1 -- 172.21.15.106:0/3510684457 <== mon.1 v2:172.21.15.171:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7fe030006330 con 0x7fe058108fc0 2024-04-28T13:06:47.692 INFO:teuthology.orchestra.run.smithi106.stderr:Error ETIMEDOUT: Connection timed out. Please check the client's permission and connection. 2024-04-28T13:06:47.715 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-04-28T13:06:47.716 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi106 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-04-28T13:06:47.971 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1888f5ee7d9b4e26a97acb041ba7abf9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi106 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-04-28T13:06:47.973 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-28T13:06:47.983 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-28T13:06:47.983 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T13:06:47.983 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:06:48.043 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-28T13:06:48.043 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-28T13:06:48.095 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:06:48.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.304+0000 7f94db200640 1 -- 172.21.15.106:0/2051971371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94d4104990 msgr2=0x7f94d4104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.304+0000 7f94db200640 1 --2- 172.21.15.106:0/2051971371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94d4104990 0x7f94d4104db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009900 tx=0x7f94c802a3f0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.304+0000 7f94db200640 1 -- 172.21.15.106:0/2051971371 shutdown_connections 2024-04-28T13:06:48.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.304+0000 7f94db200640 1 --2- 172.21.15.106:0/2051971371 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94d4106540 0x7f94d410cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.304+0000 7f94db200640 1 --2- 172.21.15.106:0/2051971371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94d4105b80 0x7f94d4106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.304+0000 7f94db200640 1 --2- 172.21.15.106:0/2051971371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94d4104990 0x7f94d4104db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.304+0000 7f94db200640 1 -- 172.21.15.106:0/2051971371 >> 172.21.15.106:0/2051971371 conn(0x7f94d40fff00 msgr2=0x7f94d4102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:48.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.304+0000 7f94db200640 1 -- 172.21.15.106:0/2051971371 shutdown_connections 2024-04-28T13:06:48.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.304+0000 7f94db200640 1 -- 172.21.15.106:0/2051971371 wait complete. 2024-04-28T13:06:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.305+0000 7f94db200640 1 Processor -- start 2024-04-28T13:06:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.305+0000 7f94db200640 1 -- start start 2024-04-28T13:06:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94db200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94d4104990 0x7f94d419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94db200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94d4105b80 0x7f94d419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94db200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94d4106540 0x7f94d41a2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94db200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4110490 con 0x7f94d4105b80 2024-04-28T13:06:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94db200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4110310 con 0x7f94d4106540 2024-04-28T13:06:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94db200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94d4110610 con 0x7f94d4104990 2024-04-28T13:06:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94d9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94d4105b80 0x7f94d419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94d9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94d4105b80 0x7f94d419b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44814/0 (socket says 172.21.15.106:44814) 2024-04-28T13:06:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94d9400640 1 -- 172.21.15.106:0/815889046 learned_addr learned my addr 172.21.15.106:0/815889046 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94d9e00640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94d4104990 0x7f94d419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94da800640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94d4106540 0x7f94d41a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94d9400640 1 -- 172.21.15.106:0/815889046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94d4104990 msgr2=0x7f94d419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94d9400640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94d4104990 0x7f94d419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94d9400640 1 -- 172.21.15.106:0/815889046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94d4106540 msgr2=0x7f94d41a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94d9400640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94d4106540 0x7f94d41a2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.306+0000 7f94d9400640 1 -- 172.21.15.106:0/815889046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94d41a27b0 con 0x7f94d4105b80 2024-04-28T13:06:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.307+0000 7f94d9400640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94d4105b80 0x7f94d419b0d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e9f0 tx=0x7f94c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.307+0000 7f94cea00640 1 -- 172.21.15.106:0/815889046 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400cd80 con 0x7f94d4105b80 2024-04-28T13:06:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.307+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94d41a2aa0 con 0x7f94d4105b80 2024-04-28T13:06:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.307+0000 7f94cea00640 1 -- 172.21.15.106:0/815889046 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94c40108e0 con 0x7f94d4105b80 2024-04-28T13:06:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.307+0000 7f94cea00640 1 -- 172.21.15.106:0/815889046 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c4018980 con 0x7f94d4105b80 2024-04-28T13:06:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.307+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94d41a2ff0 con 0x7f94d4105b80 2024-04-28T13:06:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.308+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94d410bdd0 con 0x7f94d4105b80 2024-04-28T13:06:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.310+0000 7f94cea00640 1 -- 172.21.15.106:0/815889046 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f94c4010430 con 0x7f94d4105b80 2024-04-28T13:06:48.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.312+0000 7f94cea00640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94bc0a3c40 0x7f94bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:48.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.312+0000 7f94cea00640 1 -- 172.21.15.106:0/815889046 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f94c40d0c70 con 0x7f94d4105b80 2024-04-28T13:06:48.311 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.312+0000 7f94d9e00640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94bc0a3c40 0x7f94bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:48.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.313+0000 7f94d9e00640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94bc0a3c40 0x7f94bc0a6100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009230 tx=0x7f94c80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:48.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.317+0000 7f94cea00640 1 -- 172.21.15.106:0/815889046 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f94c409ca90 con 0x7f94d4105b80 2024-04-28T13:06:48.520 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.520+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94d402dcf0 con 0x7f94d4105b80 2024-04-28T13:06:48.524 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.525+0000 7f94cea00640 1 -- 172.21.15.106:0/815889046 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24928 (secure 0 0 0) 0x7f94c40a0d90 con 0x7f94d4105b80 2024-04-28T13:06:48.525 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T13:06:48.525 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":93,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:50:58.139109+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:06:48.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.527+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94bc0a3c40 msgr2=0x7f94bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.527+0000 7f94db200640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94bc0a3c40 0x7f94bc0a6100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009230 tx=0x7f94c80314a0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.527+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94d4105b80 msgr2=0x7f94d419b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.527 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.527+0000 7f94db200640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94d4105b80 0x7f94d419b0d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f94c400e9f0 tx=0x7f94c400eeb0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.528+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 shutdown_connections 2024-04-28T13:06:48.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.528+0000 7f94db200640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f94bc0a3c40 0x7f94bc0a6100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.528+0000 7f94db200640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94d4105b80 0x7f94d419b0d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.528+0000 7f94db200640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94d4106540 0x7f94d41a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.528+0000 7f94db200640 1 --2- 172.21.15.106:0/815889046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94d4104990 0x7f94d419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.528+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 >> 172.21.15.106:0/815889046 conn(0x7f94d40fff00 msgr2=0x7f94d4108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:48.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.528+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 shutdown_connections 2024-04-28T13:06:48.528 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.528+0000 7f94db200640 1 -- 172.21.15.106:0/815889046 wait complete. 2024-04-28T13:06:48.543 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:48 smithi106 ceph-mon[28081]: pgmap v858: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:48.546 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-28T12:43:40.059873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-28T12:48:52.916511+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-28T12:48:53.528808+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-28T12:48:57.154563+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-28T13:06:48.546 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-28T13:06:48.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:48 smithi171 ceph-mon[63127]: pgmap v858: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:48.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.755+0000 7f97b3200640 1 -- 172.21.15.106:0/433351685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97b40768f0 msgr2=0x7f97b4074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.755+0000 7f97b3200640 1 --2- 172.21.15.106:0/433351685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97b40768f0 0x7f97b4074d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009900 tx=0x7f97a402a480 comp rx=0 tx=0).stop 2024-04-28T13:06:48.754 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.755+0000 7f97b3200640 1 -- 172.21.15.106:0/433351685 shutdown_connections 2024-04-28T13:06:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.755+0000 7f97b3200640 1 --2- 172.21.15.106:0/433351685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b4075310 0x7f97b4075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.755+0000 7f97b3200640 1 --2- 172.21.15.106:0/433351685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97b410a6f0 0x7f97b410cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.755+0000 7f97b3200640 1 --2- 172.21.15.106:0/433351685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97b40768f0 0x7f97b4074d40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.755+0000 7f97b3200640 1 -- 172.21.15.106:0/433351685 >> 172.21.15.106:0/433351685 conn(0x7f97b40fddd0 msgr2=0x7f97b41001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.755+0000 7f97b3200640 1 -- 172.21.15.106:0/433351685 shutdown_connections 2024-04-28T13:06:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.755+0000 7f97b3200640 1 -- 172.21.15.106:0/433351685 wait complete. 2024-04-28T13:06:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.756+0000 7f97b3200640 1 Processor -- start 2024-04-28T13:06:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.756+0000 7f97b3200640 1 -- start start 2024-04-28T13:06:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b3200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b4075310 0x7f97b419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:48.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97b40768f0 0x7f97b419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:48.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b3200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97b410a6f0 0x7f97b41a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:48.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b3200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41105c0 con 0x7f97b4075310 2024-04-28T13:06:48.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b3200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4110440 con 0x7f97b410a6f0 2024-04-28T13:06:48.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4110740 con 0x7f97b40768f0 2024-04-28T13:06:48.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b1e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b4075310 0x7f97b419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:48.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b1e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b4075310 0x7f97b419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44830/0 (socket says 172.21.15.106:44830) 2024-04-28T13:06:48.758 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b1e00640 1 -- 172.21.15.106:0/3796187698 learned_addr learned my addr 172.21.15.106:0/3796187698 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97abe00640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97b40768f0 0x7f97b419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b2800640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97b410a6f0 0x7f97b41a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b1e00640 1 -- 172.21.15.106:0/3796187698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97b40768f0 msgr2=0x7f97b419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b1e00640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97b40768f0 0x7f97b419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b1e00640 1 -- 172.21.15.106:0/3796187698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97b410a6f0 msgr2=0x7f97b41a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b1e00640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97b410a6f0 0x7f97b41a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.757+0000 7f97b1e00640 1 -- 172.21.15.106:0/3796187698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b41a28c0 con 0x7f97b4075310 2024-04-28T13:06:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.758+0000 7f97b1e00640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b4075310 0x7f97b419ac80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005ab0 tx=0x7f97a4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.758+0000 7f97aaa00640 1 -- 172.21.15.106:0/3796187698 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a403f070 con 0x7f97b4075310 2024-04-28T13:06:48.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.758+0000 7f97aaa00640 1 -- 172.21.15.106:0/3796187698 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97a4031c50 con 0x7f97b4075310 2024-04-28T13:06:48.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.758+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b41a2b50 con 0x7f97b4075310 2024-04-28T13:06:48.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.758+0000 7f97aaa00640 1 -- 172.21.15.106:0/3796187698 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4035030 con 0x7f97b4075310 2024-04-28T13:06:48.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.758+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b41a3040 con 0x7f97b4075310 2024-04-28T13:06:48.760 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.759+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b41094b0 con 0x7f97b4075310 2024-04-28T13:06:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.761+0000 7f97aaa00640 1 -- 172.21.15.106:0/3796187698 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f97a4042070 con 0x7f97b4075310 2024-04-28T13:06:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.763+0000 7f97aaa00640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f97980a3c40 0x7f97980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.763+0000 7f97aaa00640 1 -- 172.21.15.106:0/3796187698 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f97a40f1820 con 0x7f97b4075310 2024-04-28T13:06:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.763+0000 7f97abe00640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f97980a3c40 0x7f97980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.764+0000 7f97abe00640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f97980a3c40 0x7f97980a6100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f97b419c180 tx=0x7f979c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.765+0000 7f97aaa00640 1 -- 172.21.15.106:0/3796187698 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f97a4031e00 con 0x7f97b4075310 2024-04-28T13:06:48.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:48 smithi129 ceph-mon[62418]: pgmap v858: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-28T13:06:48.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.966+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97b402dcd0 con 0x7f97b4075310 2024-04-28T13:06:48.966 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.967+0000 7f97aaa00640 1 -- 172.21.15.106:0/3796187698 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f97a40b9030 con 0x7f97b4075310 2024-04-28T13:06:48.967 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2024-04-28T13:06:48.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f97980a3c40 msgr2=0x7f97980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f97980a3c40 0x7f97980a6100 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f97b419c180 tx=0x7f979c005f90 comp rx=0 tx=0).stop 2024-04-28T13:06:48.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b4075310 msgr2=0x7f97b419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:48.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b4075310 0x7f97b419ac80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f97a4005ab0 tx=0x7f97a4003b90 comp rx=0 tx=0).stop 2024-04-28T13:06:48.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 shutdown_connections 2024-04-28T13:06:48.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f97980a3c40 0x7f97980a6100 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.970 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97b410a6f0 0x7f97b41a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97b40768f0 0x7f97b419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 --2- 172.21.15.106:0/3796187698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b4075310 0x7f97b419ac80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:48.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 >> 172.21.15.106:0/3796187698 conn(0x7f97b40fddd0 msgr2=0x7f97b4106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:48.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 shutdown_connections 2024-04-28T13:06:48.971 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:48.970+0000 7f97b3200640 1 -- 172.21.15.106:0/3796187698 wait complete. 2024-04-28T13:06:48.988 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-28T13:06:49.190 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.190+0000 7f0f14e00640 1 -- 172.21.15.106:0/934044468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f101044a0 msgr2=0x7f0f1010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.190+0000 7f0f14e00640 1 --2- 172.21.15.106:0/934044468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f101044a0 0x7f0f1010ad30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009900 tx=0x7f0f0802a3f0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.191+0000 7f0f14e00640 1 -- 172.21.15.106:0/934044468 shutdown_connections 2024-04-28T13:06:49.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.191+0000 7f0f14e00640 1 --2- 172.21.15.106:0/934044468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f101044a0 0x7f0f1010ad30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.191+0000 7f0f14e00640 1 --2- 172.21.15.106:0/934044468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f10103ae0 0x7f0f10103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.191+0000 7f0f14e00640 1 --2- 172.21.15.106:0/934044468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f10100b10 0x7f0f10100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.191+0000 7f0f14e00640 1 -- 172.21.15.106:0/934044468 >> 172.21.15.106:0/934044468 conn(0x7f0f100fc460 msgr2=0x7f0f100fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:49.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.191+0000 7f0f14e00640 1 -- 172.21.15.106:0/934044468 shutdown_connections 2024-04-28T13:06:49.191 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.191+0000 7f0f14e00640 1 -- 172.21.15.106:0/934044468 wait complete. 2024-04-28T13:06:49.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.192+0000 7f0f14e00640 1 Processor -- start 2024-04-28T13:06:49.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.192+0000 7f0f14e00640 1 -- start start 2024-04-28T13:06:49.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.192+0000 7f0f14e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f10100b10 0x7f0f1019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:49.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.192+0000 7f0f14e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f10103ae0 0x7f0f1019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:49.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.192+0000 7f0f14e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f101044a0 0x7f0f101a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:49.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f14e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1010e3a0 con 0x7f0f10100b10 2024-04-28T13:06:49.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f14e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1010e220 con 0x7f0f10103ae0 2024-04-28T13:06:49.194 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f14e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1010e520 con 0x7f0f101044a0 2024-04-28T13:06:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f0f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f10100b10 0x7f0f1019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f06a00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f10103ae0 0x7f0f1019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f0f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f10100b10 0x7f0f1019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44326/0 (socket says 172.21.15.106:44326) 2024-04-28T13:06:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f0f400640 1 -- 172.21.15.106:0/927762503 learned_addr learned my addr 172.21.15.106:0/927762503 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f0fe00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f101044a0 0x7f0f101a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f06a00640 1 -- 172.21.15.106:0/927762503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f101044a0 msgr2=0x7f0f101a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f06a00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f101044a0 0x7f0f101a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f06a00640 1 -- 172.21.15.106:0/927762503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f10100b10 msgr2=0x7f0f1019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f06a00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f10100b10 0x7f0f1019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.193+0000 7f0f06a00640 1 -- 172.21.15.106:0/927762503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f101a2890 con 0x7f0f10103ae0 2024-04-28T13:06:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.194+0000 7f0f06a00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f10103ae0 0x7f0f1019b210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800e9f0 tx=0x7f0ef800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.194+0000 7f0f0cc00640 1 -- 172.21.15.106:0/927762503 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef800ccc0 con 0x7f0f10103ae0 2024-04-28T13:06:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.194+0000 7f0f0cc00640 1 -- 172.21.15.106:0/927762503 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef8005020 con 0x7f0f10103ae0 2024-04-28T13:06:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.194+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f101a2b80 con 0x7f0f10103ae0 2024-04-28T13:06:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.194+0000 7f0f0cc00640 1 -- 172.21.15.106:0/927762503 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8004a00 con 0x7f0f10103ae0 2024-04-28T13:06:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.194+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f101a30d0 con 0x7f0f10103ae0 2024-04-28T13:06:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.195+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f101a3300 con 0x7f0f10103ae0 2024-04-28T13:06:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.197+0000 7f0f0cc00640 1 -- 172.21.15.106:0/927762503 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0ef8004ba0 con 0x7f0f10103ae0 2024-04-28T13:06:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.198+0000 7f0f0cc00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ef40a3cc0 0x7f0ef40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.199+0000 7f0f0cc00640 1 -- 172.21.15.106:0/927762503 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f0ef80d0aa0 con 0x7f0f10103ae0 2024-04-28T13:06:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.199+0000 7f0f0f400640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ef40a3cc0 0x7f0ef40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.199+0000 7f0f0f400640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ef40a3cc0 0x7f0ef40a6180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f1019c150 tx=0x7f0f000096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:49.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.201+0000 7f0f0cc00640 1 -- 172.21.15.106:0/927762503 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f0ef809cf10 con 0x7f0f10103ae0 2024-04-28T13:06:49.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.403+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f101a34d0 con 0x7f0f10103ae0 2024-04-28T13:06:49.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.404+0000 7f0f0cc00640 1 -- 172.21.15.106:0/927762503 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f0ef809c810 con 0x7f0f10103ae0 2024-04-28T13:06:49.403 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T13:06:49.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.406+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ef40a3cc0 msgr2=0x7f0ef40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.406+0000 7f0f14e00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ef40a3cc0 0x7f0ef40a6180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0f1019c150 tx=0x7f0f000096d0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.406+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f10103ae0 msgr2=0x7f0f1019b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.406+0000 7f0f14e00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f10103ae0 0x7f0f1019b210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ef800e9f0 tx=0x7f0ef800eeb0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.407+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 shutdown_connections 2024-04-28T13:06:49.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.407+0000 7f0f14e00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f0ef40a3cc0 0x7f0ef40a6180 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.407+0000 7f0f14e00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f101044a0 0x7f0f101a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.407+0000 7f0f14e00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0f10103ae0 0x7f0f1019b210 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.407+0000 7f0f14e00640 1 --2- 172.21.15.106:0/927762503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f10100b10 0x7f0f1019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.407+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 >> 172.21.15.106:0/927762503 conn(0x7f0f100fc460 msgr2=0x7f0f101018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:49.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.407+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 shutdown_connections 2024-04-28T13:06:49.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.407+0000 7f0f14e00640 1 -- 172.21.15.106:0/927762503 wait complete. 2024-04-28T13:06:49.424 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-28T13:06:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.618+0000 7f17df200640 1 -- 172.21.15.106:0/3834299185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17e0109f80 msgr2=0x7f17e010a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.623 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.618+0000 7f17df200640 1 --2- 172.21.15.106:0/3834299185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17e0109f80 0x7f17e010a400 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009900 tx=0x7f17c802a3f0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.618+0000 7f17df200640 1 -- 172.21.15.106:0/3834299185 shutdown_connections 2024-04-28T13:06:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.618+0000 7f17df200640 1 --2- 172.21.15.106:0/3834299185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17e0109f80 0x7f17e010a400 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.618+0000 7f17df200640 1 --2- 172.21.15.106:0/3834299185 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17e010a940 0x7f17e01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.624 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.618+0000 7f17df200640 1 --2- 172.21.15.106:0/3834299185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e0108d90 0x7f17e01091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.618+0000 7f17df200640 1 -- 172.21.15.106:0/3834299185 >> 172.21.15.106:0/3834299185 conn(0x7f17e0077d30 msgr2=0x7f17e007a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.618+0000 7f17df200640 1 -- 172.21.15.106:0/3834299185 shutdown_connections 2024-04-28T13:06:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.618+0000 7f17df200640 1 -- 172.21.15.106:0/3834299185 wait complete. 2024-04-28T13:06:49.625 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.619+0000 7f17df200640 1 Processor -- start 2024-04-28T13:06:49.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17df200640 1 -- start start 2024-04-28T13:06:49.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17df200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e0108d90 0x7f17e019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:49.626 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17df200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17e0109f80 0x7f17e019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17df200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17e010a940 0x7f17e01a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17df200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0114910 con 0x7f17e0109f80 2024-04-28T13:06:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17df200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0114790 con 0x7f17e0108d90 2024-04-28T13:06:49.627 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17df200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0114a90 con 0x7f17e010a940 2024-04-28T13:06:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17d7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17e0109f80 0x7f17e019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17d7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17e0109f80 0x7f17e019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44350/0 (socket says 172.21.15.106:44350) 2024-04-28T13:06:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17dde00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e0108d90 0x7f17e019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17d7e00640 1 -- 172.21.15.106:0/3832916993 learned_addr learned my addr 172.21.15.106:0/3832916993 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:49.628 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.620+0000 7f17de800640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17e010a940 0x7f17e01a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17dde00640 1 -- 172.21.15.106:0/3832916993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17e010a940 msgr2=0x7f17e01a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17dde00640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17e010a940 0x7f17e01a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17dde00640 1 -- 172.21.15.106:0/3832916993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17e0109f80 msgr2=0x7f17e019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17dde00640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17e0109f80 0x7f17e019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.629 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17dde00640 1 -- 172.21.15.106:0/3832916993 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e01a6bd0 con 0x7f17e0108d90 2024-04-28T13:06:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17dde00640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e0108d90 0x7f17e019efb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17d000e9c0 tx=0x7f17d000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17d6a00640 1 -- 172.21.15.106:0/3832916993 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d000cd80 con 0x7f17e0108d90 2024-04-28T13:06:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17d6a00640 1 -- 172.21.15.106:0/3832916993 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d00108e0 con 0x7f17e0108d90 2024-04-28T13:06:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e01a6ec0 con 0x7f17e0108d90 2024-04-28T13:06:49.630 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17d6a00640 1 -- 172.21.15.106:0/3832916993 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d00187f0 con 0x7f17e0108d90 2024-04-28T13:06:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.621+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e01a7320 con 0x7f17e0108d90 2024-04-28T13:06:49.631 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.622+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e0110100 con 0x7f17e0108d90 2024-04-28T13:06:49.632 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.624+0000 7f17d6a00640 1 -- 172.21.15.106:0/3832916993 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f17d001f020 con 0x7f17e0108d90 2024-04-28T13:06:49.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.626+0000 7f17d6a00640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17c40a3c40 0x7f17c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:49.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.626+0000 7f17d6a00640 1 -- 172.21.15.106:0/3832916993 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f17d00d09a0 con 0x7f17e0108d90 2024-04-28T13:06:49.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.626+0000 7f17d7e00640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17c40a3c40 0x7f17c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:49.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.627+0000 7f17d7e00640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17c40a3c40 0x7f17c40a6100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f17e01a04c0 tx=0x7f17c8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:49.633 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.630+0000 7f17d6a00640 1 -- 172.21.15.106:0/3832916993 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f17d009ce10 con 0x7f17e0108d90 2024-04-28T13:06:49.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:49 smithi171 ceph-mon[63127]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T13:06:49.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:49 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/815889046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T13:06:49.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:49 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3796187698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T13:06:49.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:49 smithi129 ceph-mon[62418]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T13:06:49.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:49 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/815889046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T13:06:49.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:49 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3796187698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T13:06:49.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:49 smithi106 ceph-mon[28081]: from='client.25456 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-28T13:06:49.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:49 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/815889046' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T13:06:49.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:49 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3796187698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-28T13:06:49.831 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.831+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f17e002dd30 con 0x7f17e0108d90 2024-04-28T13:06:49.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.832+0000 7f17d6a00640 1 -- 172.21.15.106:0/3832916993 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f17d009c710 con 0x7f17e0108d90 2024-04-28T13:06:49.833 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2024-04-28T13:06:49.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17c40a3c40 msgr2=0x7f17c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17c40a3c40 0x7f17c40a6100 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f17e01a04c0 tx=0x7f17c8005a20 comp rx=0 tx=0).stop 2024-04-28T13:06:49.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e0108d90 msgr2=0x7f17e019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:49.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e0108d90 0x7f17e019efb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f17d000e9c0 tx=0x7f17d000ee80 comp rx=0 tx=0).stop 2024-04-28T13:06:49.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 shutdown_connections 2024-04-28T13:06:49.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f17c40a3c40 0x7f17c40a6100 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17e0109f80 0x7f17e019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17e010a940 0x7f17e01a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 --2- 172.21.15.106:0/3832916993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17e0108d90 0x7f17e019efb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:49.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 >> 172.21.15.106:0/3832916993 conn(0x7f17e0077d30 msgr2=0x7f17e010d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:49.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 shutdown_connections 2024-04-28T13:06:49.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:49.835+0000 7f17df200640 1 -- 172.21.15.106:0/3832916993 wait complete. 2024-04-28T13:06:49.853 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-28T13:06:50.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.047+0000 7f3cb9200640 1 -- 172.21.15.106:0/249217665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb4108f90 msgr2=0x7f3cb41093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.047+0000 7f3cb9200640 1 --2- 172.21.15.106:0/249217665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb4108f90 0x7f3cb41093b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009860 tx=0x7f3ca402a340 comp rx=0 tx=0).stop 2024-04-28T13:06:50.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.047+0000 7f3cb9200640 1 -- 172.21.15.106:0/249217665 shutdown_connections 2024-04-28T13:06:50.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.047+0000 7f3cb9200640 1 --2- 172.21.15.106:0/249217665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb410a180 0x7f3cb410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.048 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.047+0000 7f3cb9200640 1 --2- 172.21.15.106:0/249217665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cb410ab40 0x7f3cb41113d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.047+0000 7f3cb9200640 1 --2- 172.21.15.106:0/249217665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb4108f90 0x7f3cb41093b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.047+0000 7f3cb9200640 1 -- 172.21.15.106:0/249217665 >> 172.21.15.106:0/249217665 conn(0x7f3cb4077f30 msgr2=0x7f3cb407a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:50.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.047+0000 7f3cb9200640 1 -- 172.21.15.106:0/249217665 shutdown_connections 2024-04-28T13:06:50.049 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.047+0000 7f3cb9200640 1 -- 172.21.15.106:0/249217665 wait complete. 2024-04-28T13:06:50.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.048+0000 7f3cb9200640 1 Processor -- start 2024-04-28T13:06:50.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb9200640 1 -- start start 2024-04-28T13:06:50.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb9200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb4108f90 0x7f3cb419f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:50.051 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb9200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb410a180 0x7f3cb419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb9200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cb410ab40 0x7f3cb41a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb9200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4114a90 con 0x7f3cb410a180 2024-04-28T13:06:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb9200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4114910 con 0x7f3cb4108f90 2024-04-28T13:06:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb9200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb4114c10 con 0x7f3cb410ab40 2024-04-28T13:06:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb410a180 0x7f3cb419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb4108f90 0x7f3cb419f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb410a180 0x7f3cb419f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44360/0 (socket says 172.21.15.106:44360) 2024-04-28T13:06:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb1400640 1 -- 172.21.15.106:0/3650202591 learned_addr learned my addr 172.21.15.106:0/3650202591 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.049+0000 7f3cb2800640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cb410ab40 0x7f3cb41a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.050+0000 7f3cb1e00640 1 -- 172.21.15.106:0/3650202591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cb410ab40 msgr2=0x7f3cb41a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.050+0000 7f3cb1e00640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cb410ab40 0x7f3cb41a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.050+0000 7f3cb1e00640 1 -- 172.21.15.106:0/3650202591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb410a180 msgr2=0x7f3cb419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.050+0000 7f3cb1e00640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb410a180 0x7f3cb419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.050+0000 7f3cb1e00640 1 -- 172.21.15.106:0/3650202591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb41a6e10 con 0x7f3cb4108f90 2024-04-28T13:06:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.050+0000 7f3cb1e00640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb4108f90 0x7f3cb419f240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ca402a860 tx=0x7f3ca4032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.050+0000 7f3caaa00640 1 -- 172.21.15.106:0/3650202591 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4040020 con 0x7f3cb4108f90 2024-04-28T13:06:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.050+0000 7f3caaa00640 1 -- 172.21.15.106:0/3650202591 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca4030d50 con 0x7f3cb4108f90 2024-04-28T13:06:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.051+0000 7f3caaa00640 1 -- 172.21.15.106:0/3650202591 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4037040 con 0x7f3cb4108f90 2024-04-28T13:06:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.051+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb41a70a0 con 0x7f3cb4108f90 2024-04-28T13:06:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.051+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb41a7590 con 0x7f3cb4108f90 2024-04-28T13:06:50.055 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.052+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb41103d0 con 0x7f3cb4108f90 2024-04-28T13:06:50.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.054+0000 7f3caaa00640 1 -- 172.21.15.106:0/3650202591 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3ca4043020 con 0x7f3cb4108f90 2024-04-28T13:06:50.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.055+0000 7f3caaa00640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3c980a3c40 0x7f3c980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:50.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.055+0000 7f3caaa00640 1 -- 172.21.15.106:0/3650202591 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f3ca40f0bb0 con 0x7f3cb4108f90 2024-04-28T13:06:50.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.055+0000 7f3cb1400640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3c980a3c40 0x7f3c980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:50.056 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.056+0000 7f3cb1400640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3c980a3c40 0x7f3c980a6100 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3cb41a0700 tx=0x7f3ca0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:50.057 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.058+0000 7f3caaa00640 1 -- 172.21.15.106:0/3650202591 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f3ca404a020 con 0x7f3cb4108f90 2024-04-28T13:06:50.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.261+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3cb402dc60 con 0x7f3cb4108f90 2024-04-28T13:06:50.262 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.262+0000 7f3caaa00640 1 -- 172.21.15.106:0/3650202591 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f3ca40bd130 con 0x7f3cb4108f90 2024-04-28T13:06:50.263 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2024-04-28T13:06:50.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.265+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3c980a3c40 msgr2=0x7f3c980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.265+0000 7f3cb9200640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3c980a3c40 0x7f3c980a6100 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3cb41a0700 tx=0x7f3ca0005f90 comp rx=0 tx=0).stop 2024-04-28T13:06:50.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.265+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb4108f90 msgr2=0x7f3cb419f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.265+0000 7f3cb9200640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb4108f90 0x7f3cb419f240 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3ca402a860 tx=0x7f3ca4032d50 comp rx=0 tx=0).stop 2024-04-28T13:06:50.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.266+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 shutdown_connections 2024-04-28T13:06:50.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.266+0000 7f3cb9200640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f3c980a3c40 0x7f3c980a6100 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.266+0000 7f3cb9200640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb410a180 0x7f3cb419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.266+0000 7f3cb9200640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cb410ab40 0x7f3cb41a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.266+0000 7f3cb9200640 1 --2- 172.21.15.106:0/3650202591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3cb4108f90 0x7f3cb419f240 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.266+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 >> 172.21.15.106:0/3650202591 conn(0x7f3cb4077f30 msgr2=0x7f3cb410d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:50.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.266+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 shutdown_connections 2024-04-28T13:06:50.268 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.266+0000 7f3cb9200640 1 -- 172.21.15.106:0/3650202591 wait complete. 2024-04-28T13:06:50.286 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-28T13:06:50.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.485+0000 7fbe8de00640 1 -- 172.21.15.106:0/2691399774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe88105b80 msgr2=0x7fbe88106000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.485+0000 7fbe8de00640 1 --2- 172.21.15.106:0/2691399774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe88105b80 0x7fbe88106000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009860 tx=0x7fbe7002a340 comp rx=0 tx=0).stop 2024-04-28T13:06:50.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.486+0000 7fbe8de00640 1 -- 172.21.15.106:0/2691399774 shutdown_connections 2024-04-28T13:06:50.486 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.486+0000 7fbe8de00640 1 --2- 172.21.15.106:0/2691399774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe88106540 0x7fbe8810cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.486+0000 7fbe8de00640 1 --2- 172.21.15.106:0/2691399774 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe88105b80 0x7fbe88106000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.486+0000 7fbe8de00640 1 --2- 172.21.15.106:0/2691399774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe88104990 0x7fbe88104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.486+0000 7fbe8de00640 1 -- 172.21.15.106:0/2691399774 >> 172.21.15.106:0/2691399774 conn(0x7fbe880fff00 msgr2=0x7fbe88102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:50.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.486+0000 7fbe8de00640 1 -- 172.21.15.106:0/2691399774 shutdown_connections 2024-04-28T13:06:50.487 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.486+0000 7fbe8de00640 1 -- 172.21.15.106:0/2691399774 wait complete. 2024-04-28T13:06:50.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8de00640 1 Processor -- start 2024-04-28T13:06:50.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8de00640 1 -- start start 2024-04-28T13:06:50.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe88104990 0x7fbe8819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:50.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8de00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe88105b80 0x7fbe8819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:50.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8de00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe88106540 0x7fbe881a2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:50.489 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8de00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe881104a0 con 0x7fbe88104990 2024-04-28T13:06:50.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8de00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe88110320 con 0x7fbe88105b80 2024-04-28T13:06:50.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8de00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbe88110620 con 0x7fbe88106540 2024-04-28T13:06:50.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe88104990 0x7fbe8819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:50.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe88104990 0x7fbe8819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44382/0 (socket says 172.21.15.106:44382) 2024-04-28T13:06:50.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8ca00640 1 -- 172.21.15.106:0/3936029136 learned_addr learned my addr 172.21.15.106:0/3936029136 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:50.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe8d400640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe88106540 0x7fbe881a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:50.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.487+0000 7fbe7fe00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe88105b80 0x7fbe8819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:50.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe8ca00640 1 -- 172.21.15.106:0/3936029136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe88106540 msgr2=0x7fbe881a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe8ca00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe88106540 0x7fbe881a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe8ca00640 1 -- 172.21.15.106:0/3936029136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe88105b80 msgr2=0x7fbe8819b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe8ca00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe88105b80 0x7fbe8819b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe8ca00640 1 -- 172.21.15.106:0/3936029136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe881a2790 con 0x7fbe88104990 2024-04-28T13:06:50.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe8ca00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe88104990 0x7fbe8819ab70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c3a0 tx=0x7fbe7800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:50.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe86000640 1 -- 172.21.15.106:0/3936029136 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78003970 con 0x7fbe88104990 2024-04-28T13:06:50.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe86000640 1 -- 172.21.15.106:0/3936029136 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe78004020 con 0x7fbe88104990 2024-04-28T13:06:50.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe86000640 1 -- 172.21.15.106:0/3936029136 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800b920 con 0x7fbe88104990 2024-04-28T13:06:50.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.488+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe881a2a80 con 0x7fbe88104990 2024-04-28T13:06:50.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.489+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe881a2fa0 con 0x7fbe88104990 2024-04-28T13:06:50.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.489+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8810bd50 con 0x7fbe88104990 2024-04-28T13:06:50.492 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.491+0000 7fbe86000640 1 -- 172.21.15.106:0/3936029136 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbe7801d020 con 0x7fbe88104990 2024-04-28T13:06:50.493 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.494+0000 7fbe86000640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:50.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.494+0000 7fbe86000640 1 -- 172.21.15.106:0/3936029136 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fbe780d09e0 con 0x7fbe88104990 2024-04-28T13:06:50.494 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.494+0000 7fbe7fe00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:50.495 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.495+0000 7fbe7fe00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbe7002a930 tx=0x7fbe70030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:50.499 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.498+0000 7fbe86000640 1 -- 172.21.15.106:0/3936029136 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbe780041d0 con 0x7fbe88104990 2024-04-28T13:06:50.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:50 smithi106 ceph-mon[28081]: pgmap v859: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:50.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:50 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/927762503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T13:06:50.579 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:50 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3832916993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T13:06:50.580 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:50 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3650202591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T13:06:50.651 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:50 smithi171 ceph-mon[63127]: pgmap v859: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:50.651 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:50 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/927762503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T13:06:50.651 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:50 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3832916993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T13:06:50.651 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:50 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3650202591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T13:06:50.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.716+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbe8802dad0 con 0x7fbe88104990 2024-04-28T13:06:50.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.717+0000 7fbe86000640 1 -- 172.21.15.106:0/3936029136 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+599 (secure 0 0 0) 0x7fbe7809ce50 con 0x7fbe88104990 2024-04-28T13:06:50.719 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6801/2253669851 2024-04-29T12:40:55.904029+0000 2024-04-28T13:06:50.719 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6800/2253669851 2024-04-29T12:40:55.904029+0000 2024-04-28T13:06:50.720 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/3545890663 2024-04-29T12:40:55.904029+0000 2024-04-28T13:06:50.720 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/1369056801 2024-04-29T12:40:55.904029+0000 2024-04-28T13:06:50.720 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/938762297 2024-04-29T12:40:21.812705+0000 2024-04-28T13:06:50.720 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/1467729753 2024-04-29T12:40:21.812705+0000 2024-04-28T13:06:50.720 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6800/102356115 2024-04-29T12:40:21.812705+0000 2024-04-28T13:06:50.720 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/2991131145 2024-04-29T12:40:21.812705+0000 2024-04-28T13:06:50.720 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:6801/102356115 2024-04-29T12:40:21.812705+0000 2024-04-28T13:06:50.721 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106:0/2350345760 2024-04-29T12:40:55.904029+0000 2024-04-28T13:06:50.722 INFO:teuthology.orchestra.run.smithi106.stderr:listed 10 entries 2024-04-28T13:06:50.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbe6c0a3c40 msgr2=0x7fbe6c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbe7002a930 tx=0x7fbe70030450 comp rx=0 tx=0).stop 2024-04-28T13:06:50.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe88104990 msgr2=0x7fbe8819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:50.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe88104990 0x7fbe8819ab70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c3a0 tx=0x7fbe7800c860 comp rx=0 tx=0).stop 2024-04-28T13:06:50.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 shutdown_connections 2024-04-28T13:06:50.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbe6c0a3c40 0x7fbe6c0a6100 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbe88106540 0x7fbe881a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe88105b80 0x7fbe8819b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 --2- 172.21.15.106:0/3936029136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe88104990 0x7fbe8819ab70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:50.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 >> 172.21.15.106:0/3936029136 conn(0x7fbe880fff00 msgr2=0x7fbe88108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:50.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 shutdown_connections 2024-04-28T13:06:50.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:50.720+0000 7fbe8de00640 1 -- 172.21.15.106:0/3936029136 wait complete. 2024-04-28T13:06:50.738 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-28T13:06:50.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:06:50.738 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-28T13:06:50.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:50 smithi129 ceph-mon[62418]: pgmap v859: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-28T13:06:50.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:50 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/927762503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-28T13:06:50.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:50 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3832916993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-28T13:06:50.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:50 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3650202591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-28T13:06:50.847 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2024-04-28T13:06:50.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:06:50.848 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-28T13:06:50.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:06:50.901 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2024-04-28T13:06:50.948 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-28T13:06:50.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:06:50.949 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-28T13:06:51.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-28T13:06:51.031 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2024-04-28T13:06:51.151 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-28T13:06:51.161 INFO:tasks.cephadm:Teardown begin 2024-04-28T13:06:51.161 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-28T13:06:51.161 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:06:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.345+0000 7f3083e00640 1 -- 172.21.15.106:0/1140421897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30840772f0 msgr2=0x7f3084077710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.345+0000 7f3083e00640 1 --2- 172.21.15.106:0/1140421897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30840772f0 0x7f3084077710 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3074009900 tx=0x7f307402a3f0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.345 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.345+0000 7f3083e00640 1 -- 172.21.15.106:0/1140421897 shutdown_connections 2024-04-28T13:06:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.345+0000 7f3083e00640 1 --2- 172.21.15.106:0/1140421897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084078ed0 0x7f3084079770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.345+0000 7f3083e00640 1 --2- 172.21.15.106:0/1140421897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3084078530 0x7f3084078990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.345+0000 7f3083e00640 1 --2- 172.21.15.106:0/1140421897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30840772f0 0x7f3084077710 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.345+0000 7f3083e00640 1 -- 172.21.15.106:0/1140421897 >> 172.21.15.106:0/1140421897 conn(0x7f30840ffeb0 msgr2=0x7f3084102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.345+0000 7f3083e00640 1 -- 172.21.15.106:0/1140421897 shutdown_connections 2024-04-28T13:06:51.346 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.345+0000 7f3083e00640 1 -- 172.21.15.106:0/1140421897 wait complete. 2024-04-28T13:06:51.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.346+0000 7f3083e00640 1 Processor -- start 2024-04-28T13:06:51.348 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.346+0000 7f3083e00640 1 -- start start 2024-04-28T13:06:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.346+0000 7f3083e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30840772f0 0x7f308419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.346+0000 7f3083e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3084078530 0x7f308419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.346+0000 7f3083e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084078ed0 0x7f30841a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.346+0000 7f3083e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30841148e0 con 0x7f3084078ed0 2024-04-28T13:06:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.346+0000 7f3083e00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3084114760 con 0x7f3084078530 2024-04-28T13:06:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.346+0000 7f3083e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3084114a60 con 0x7f30840772f0 2024-04-28T13:06:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.346+0000 7f3083400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084078ed0 0x7f30841a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3083400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084078ed0 0x7f30841a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44392/0 (socket says 172.21.15.106:44392) 2024-04-28T13:06:51.349 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3083400640 1 -- 172.21.15.106:0/2979255755 learned_addr learned my addr 172.21.15.106:0/2979255755 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3082a00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30840772f0 0x7f308419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3082000640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3084078530 0x7f308419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3083400640 1 -- 172.21.15.106:0/2979255755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30840772f0 msgr2=0x7f308419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3083400640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30840772f0 0x7f308419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3083400640 1 -- 172.21.15.106:0/2979255755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3084078530 msgr2=0x7f308419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3083400640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3084078530 0x7f308419f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3083400640 1 -- 172.21.15.106:0/2979255755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30841a6bd0 con 0x7f3084078ed0 2024-04-28T13:06:51.350 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.347+0000 7f3083400640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084078ed0 0x7f30841a6560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f307c00eb10 tx=0x7f307c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:51.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.348+0000 7f3088a00640 1 -- 172.21.15.106:0/2979255755 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c004b20 con 0x7f3084078ed0 2024-04-28T13:06:51.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.348+0000 7f3088a00640 1 -- 172.21.15.106:0/2979255755 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f307c004cc0 con 0x7f3084078ed0 2024-04-28T13:06:51.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.348+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30841a6ec0 con 0x7f3084078ed0 2024-04-28T13:06:51.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.348+0000 7f3088a00640 1 -- 172.21.15.106:0/2979255755 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c010670 con 0x7f3084078ed0 2024-04-28T13:06:51.351 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.348+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30841a7330 con 0x7f3084078ed0 2024-04-28T13:06:51.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.349+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30841112d0 con 0x7f3084078ed0 2024-04-28T13:06:51.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.351+0000 7f3088a00640 1 -- 172.21.15.106:0/2979255755 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f307c005020 con 0x7f3084078ed0 2024-04-28T13:06:51.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.352+0000 7f3088a00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f30680a3c40 0x7f30680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:51.352 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.352+0000 7f3088a00640 1 -- 172.21.15.106:0/2979255755 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f307c0d0f00 con 0x7f3084078ed0 2024-04-28T13:06:51.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.352+0000 7f3082a00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f30680a3c40 0x7f30680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:51.353 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.353+0000 7f3082a00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f30680a3c40 0x7f30680a6100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3074031500 tx=0x7f3074005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:51.358 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.358+0000 7f3088a00640 1 -- 172.21.15.106:0/2979255755 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f307c09d370 con 0x7f3084078ed0 2024-04-28T13:06:51.606 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.606+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f308402dad0 con 0x7f3084078ed0 2024-04-28T13:06:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.607+0000 7f3088a00640 1 -- 172.21.15.106:0/2979255755 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11953 (secure 0 0 0) 0x7f307c09cc70 con 0x7f3084078ed0 2024-04-28T13:06:51.608 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T13:06:51.608 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:51:46.476352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24548,"mds_3":24538,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24566,24538,24578,24548,24520]},"id":1}]} 2024-04-28T13:06:51.609 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 27 2024-04-28T13:06:51.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.609+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f30680a3c40 msgr2=0x7f30680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:51.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.609+0000 7f3083e00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f30680a3c40 0x7f30680a6100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3074031500 tx=0x7f3074005a20 comp rx=0 tx=0).stop 2024-04-28T13:06:51.610 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.609+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084078ed0 msgr2=0x7f30841a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.609+0000 7f3083e00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084078ed0 0x7f30841a6560 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f307c00eb10 tx=0x7f307c00c2d0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.610+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 shutdown_connections 2024-04-28T13:06:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.610+0000 7f3083e00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f30680a3c40 0x7f30680a6100 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.610+0000 7f3083e00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084078ed0 0x7f30841a6560 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.610+0000 7f3083e00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3084078530 0x7f308419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.610+0000 7f3083e00640 1 --2- 172.21.15.106:0/2979255755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f30840772f0 0x7f308419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.610+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 >> 172.21.15.106:0/2979255755 conn(0x7f30840ffeb0 msgr2=0x7f30841019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.610+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 shutdown_connections 2024-04-28T13:06:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.610+0000 7f3083e00640 1 -- 172.21.15.106:0/2979255755 wait complete. 2024-04-28T13:06:51.626 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:51 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3936029136' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-28T13:06:51.630 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-28T13:06:51.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:51 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3936029136' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-28T13:06:51.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:51 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3936029136' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.831+0000 7f4c29c00640 1 -- 172.21.15.106:0/2916273221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 msgr2=0x7f4c2410cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.831+0000 7f4c29c00640 1 --2- 172.21.15.106:0/2916273221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 0x7f4c2410cdc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009ca0 tx=0x7f4c1c02a8c0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.831+0000 7f4c29c00640 1 -- 172.21.15.106:0/2916273221 shutdown_connections 2024-04-28T13:06:51.832 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.831+0000 7f4c29c00640 1 --2- 172.21.15.106:0/2916273221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 0x7f4c2410cdc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.831+0000 7f4c29c00640 1 --2- 172.21.15.106:0/2916273221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c24105b70 0x7f4c24105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.831+0000 7f4c29c00640 1 --2- 172.21.15.106:0/2916273221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c24104980 0x7f4c24104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.831+0000 7f4c29c00640 1 -- 172.21.15.106:0/2916273221 >> 172.21.15.106:0/2916273221 conn(0x7f4c240fff10 msgr2=0x7f4c24102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.831+0000 7f4c29c00640 1 -- 172.21.15.106:0/2916273221 shutdown_connections 2024-04-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.831+0000 7f4c29c00640 1 -- 172.21.15.106:0/2916273221 wait complete. 2024-04-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.832+0000 7f4c29c00640 1 Processor -- start 2024-04-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.832+0000 7f4c29c00640 1 -- start start 2024-04-28T13:06:51.833 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.832+0000 7f4c29c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c24104980 0x7f4c2419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.832+0000 7f4c29c00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c24105b70 0x7f4c2419b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.832+0000 7f4c29c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 0x7f4c241a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.832+0000 7f4c29c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24110500 con 0x7f4c24106530 2024-04-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.832+0000 7f4c29c00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24110380 con 0x7f4c24105b70 2024-04-28T13:06:51.834 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.832+0000 7f4c29c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24110680 con 0x7f4c24104980 2024-04-28T13:06:51.835 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c29200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 0x7f4c241a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c29200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 0x7f4c241a20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44414/0 (socket says 172.21.15.106:44414) 2024-04-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c29200640 1 -- 172.21.15.106:0/3115555963 learned_addr learned my addr 172.21.15.106:0/3115555963 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c1be00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c24105b70 0x7f4c2419b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c23e00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c24104980 0x7f4c2419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c29200640 1 -- 172.21.15.106:0/3115555963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c24104980 msgr2=0x7f4c2419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c29200640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c24104980 0x7f4c2419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c29200640 1 -- 172.21.15.106:0/3115555963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c24105b70 msgr2=0x7f4c2419b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:51.836 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c29200640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c24105b70 0x7f4c2419b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c29200640 1 -- 172.21.15.106:0/3115555963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c241a2730 con 0x7f4c24106530 2024-04-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c29200640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 0x7f4c241a20c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c033040 tx=0x7f4c1c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.833+0000 7f4c21600640 1 -- 172.21.15.106:0/3115555963 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c003b70 con 0x7f4c24106530 2024-04-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.834+0000 7f4c21600640 1 -- 172.21.15.106:0/3115555963 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c1c003d10 con 0x7f4c24106530 2024-04-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.834+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c241a29c0 con 0x7f4c24106530 2024-04-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.834+0000 7f4c21600640 1 -- 172.21.15.106:0/3115555963 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c02fb40 con 0x7f4c24106530 2024-04-28T13:06:51.837 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.834+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c241a2f10 con 0x7f4c24106530 2024-04-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.835+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2410bc40 con 0x7f4c24106530 2024-04-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.837+0000 7f4c21600640 1 -- 172.21.15.106:0/3115555963 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4c1c004610 con 0x7f4c24106530 2024-04-28T13:06:51.838 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.838+0000 7f4c21600640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4c080a3c40 0x7f4c080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:51.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.838+0000 7f4c21600640 1 -- 172.21.15.106:0/3115555963 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f4c1c0f1de0 con 0x7f4c24106530 2024-04-28T13:06:51.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.838+0000 7f4c23e00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4c080a3c40 0x7f4c080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:51.839 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.839+0000 7f4c23e00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4c080a3c40 0x7f4c080a6100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4c241059d0 tx=0x7f4c14005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:51.842 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:51.842+0000 7f4c21600640 1 -- 172.21.15.106:0/3115555963 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f4c1c0be360 con 0x7f4c24106530 2024-04-28T13:06:52.077 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.077+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c24063530 con 0x7f4c24106530 2024-04-28T13:06:52.078 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.078+0000 7f4c21600640 1 -- 172.21.15.106:0/3115555963 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11953 (secure 0 0 0) 0x7f4c1c0dce20 con 0x7f4c24106530 2024-04-28T13:06:52.079 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T13:06:52.079 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/2537709936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":2537709936},{"type":"v1","addr":"172.21.15.129:6833","nonce":2537709936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/1724808557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":1724808557},{"type":"v1","addr":"172.21.15.171:6833","nonce":1724808557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-28T12:48:54.598061+0000","modified":"2024-04-28T12:51:46.476352+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24578,"mds_1":24566,"mds_2":24548,"mds_3":24538,"mds_4":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24469":{"gid":24469,"name":"j","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6835/3035871722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3035871722},{"type":"v1","addr":"172.21.15.106:6835","nonce":3035871722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"d","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/4131098763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":4131098763},{"type":"v1","addr":"172.21.15.106:6837","nonce":4131098763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/1242652051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":1242652051},{"type":"v1","addr":"172.21.15.129:6835","nonce":1242652051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24508":{"gid":24508,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6835/4221893526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4221893526},{"type":"v1","addr":"172.21.15.171:6835","nonce":4221893526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/1065884016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1065884016},{"type":"v1","addr":"172.21.15.129:6837","nonce":1065884016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24520":{"gid":24520,"name":"c","rank":4,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.171:6837/813345311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":813345311},{"type":"v1","addr":"172.21.15.171:6837","nonce":813345311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":3,"incarnation":22,"state":"up:active","state_seq":14,"addr":"172.21.15.171:6839/2406303763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":2406303763},{"type":"v1","addr":"172.21.15.171:6839","nonce":2406303763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"g","rank":2,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.106:6839/3863798001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3863798001},{"type":"v1","addr":"172.21.15.106:6839","nonce":3863798001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"a","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.106:6841/92504617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6840","nonce":92504617},{"type":"v1","addr":"172.21.15.106:6841","nonce":92504617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24578":{"gid":24578,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.129:6839/4208792665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":4208792665},{"type":"v1","addr":"172.21.15.129:6839","nonce":4208792665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24578,"qdb_cluster":[24566,24538,24578,24548,24520]},"id":1}]} 2024-04-28T13:06:52.079 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 27 2024-04-28T13:06:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.080+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4c080a3c40 msgr2=0x7f4c080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.080+0000 7f4c29c00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4c080a3c40 0x7f4c080a6100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f4c241059d0 tx=0x7f4c14005f90 comp rx=0 tx=0).stop 2024-04-28T13:06:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.080+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 msgr2=0x7f4c241a20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.080+0000 7f4c29c00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 0x7f4c241a20c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c033040 tx=0x7f4c1c0316f0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.081+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 shutdown_connections 2024-04-28T13:06:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.081+0000 7f4c29c00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f4c080a3c40 0x7f4c080a6100 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.081 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.081+0000 7f4c29c00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c24106530 0x7f4c241a20c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.081+0000 7f4c29c00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c24105b70 0x7f4c2419b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.081+0000 7f4c29c00640 1 --2- 172.21.15.106:0/3115555963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4c24104980 0x7f4c2419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.081+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 >> 172.21.15.106:0/3115555963 conn(0x7f4c240fff10 msgr2=0x7f4c24108e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.081+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 shutdown_connections 2024-04-28T13:06:52.082 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.081+0000 7f4c29c00640 1 -- 172.21.15.106:0/3115555963 wait complete. 2024-04-28T13:06:52.099 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-28T13:06:52.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.297+0000 7fbdad000640 1 -- 172.21.15.106:0/4129361108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8104300 msgr2=0x7fbda810ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.297+0000 7fbdad000640 1 --2- 172.21.15.106:0/4129361108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8104300 0x7fbda810ab90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009900 tx=0x7fbda002a400 comp rx=0 tx=0).stop 2024-04-28T13:06:52.298 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.297+0000 7fbdad000640 1 -- 172.21.15.106:0/4129361108 shutdown_connections 2024-04-28T13:06:52.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.297+0000 7fbdad000640 1 --2- 172.21.15.106:0/4129361108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8104300 0x7fbda810ab90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.297+0000 7fbdad000640 1 --2- 172.21.15.106:0/4129361108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8103940 0x7fbda8103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.297+0000 7fbdad000640 1 --2- 172.21.15.106:0/4129361108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbda8102750 0x7fbda8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.297+0000 7fbdad000640 1 -- 172.21.15.106:0/4129361108 >> 172.21.15.106:0/4129361108 conn(0x7fbda80fdcc0 msgr2=0x7fbda8100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:52.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.298+0000 7fbdad000640 1 -- 172.21.15.106:0/4129361108 shutdown_connections 2024-04-28T13:06:52.299 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.298+0000 7fbdad000640 1 -- 172.21.15.106:0/4129361108 wait complete. 2024-04-28T13:06:52.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbdad000640 1 Processor -- start 2024-04-28T13:06:52.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbdad000640 1 -- start start 2024-04-28T13:06:52.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbdad000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8102750 0x7fbda8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:52.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbdad000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8103940 0x7fbda8198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:52.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbdad000640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbda8104300 0x7fbda819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:52.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbdad000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810e280 con 0x7fbda8102750 2024-04-28T13:06:52.302 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbdad000640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810e100 con 0x7fbda8104300 2024-04-28T13:06:52.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbdad000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbda810e400 con 0x7fbda8103940 2024-04-28T13:06:52.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbda7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8102750 0x7fbda8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:52.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbda7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8102750 0x7fbda8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44430/0 (socket says 172.21.15.106:44430) 2024-04-28T13:06:52.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbda7400640 1 -- 172.21.15.106:0/3012308678 learned_addr learned my addr 172.21.15.106:0/3012308678 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:52.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbda7e00640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbda8104300 0x7fbda819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:52.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.299+0000 7fbd9ea00640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8103940 0x7fbda8198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:52.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbda7400640 1 -- 172.21.15.106:0/3012308678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8103940 msgr2=0x7fbda8198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.303 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbda7400640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8103940 0x7fbda8198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbda7400640 1 -- 172.21.15.106:0/3012308678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbda8104300 msgr2=0x7fbda819fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbda7400640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbda8104300 0x7fbda819fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbda7400640 1 -- 172.21.15.106:0/3012308678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda81a0540 con 0x7fbda8102750 2024-04-28T13:06:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbda7400640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8102750 0x7fbda8198920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800ebc0 tx=0x7fbd9800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbda4c00640 1 -- 172.21.15.106:0/3012308678 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98003b70 con 0x7fbda8102750 2024-04-28T13:06:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbda4c00640 1 -- 172.21.15.106:0/3012308678 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd98003d10 con 0x7fbda8102750 2024-04-28T13:06:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbda4c00640 1 -- 172.21.15.106:0/3012308678 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd98019800 con 0x7fbda8102750 2024-04-28T13:06:52.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda81a0830 con 0x7fbda8102750 2024-04-28T13:06:52.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.300+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda81a0dd0 con 0x7fbda8102750 2024-04-28T13:06:52.305 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.301+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda8109af0 con 0x7fbda8102750 2024-04-28T13:06:52.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.303+0000 7fbda4c00640 1 -- 172.21.15.106:0/3012308678 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbd980199a0 con 0x7fbda8102750 2024-04-28T13:06:52.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.305+0000 7fbda4c00640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbd8c0a3cc0 0x7fbd8c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:52.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.305+0000 7fbda4c00640 1 -- 172.21.15.106:0/3012308678 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7fbd980d11a0 con 0x7fbda8102750 2024-04-28T13:06:52.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.305+0000 7fbd9ea00640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbd8c0a3cc0 0x7fbd8c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:52.306 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.306+0000 7fbd9ea00640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbd8c0a3cc0 0x7fbd8c0a6180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbda8199e30 tx=0x7fbd90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:52.308 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.309+0000 7fbda4c00640 1 -- 172.21.15.106:0/3012308678 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fbd9809d610 con 0x7fbda8102750 2024-04-28T13:06:52.501 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.501+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbda802dd60 con 0x7fbda8102750 2024-04-28T13:06:52.505 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.504+0000 7fbda4c00640 1 -- 172.21.15.106:0/3012308678 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24928 (secure 0 0 0) 0x7fbd9809cf10 con 0x7fbda8102750 2024-04-28T13:06:52.505 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-28T13:06:52.505 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":93,"fsid":"e5e35dfa-055b-11ef-bc93-c7b262605968","created":"2024-04-28T12:39:49.393843+0000","modified":"2024-04-28T12:50:58.139109+0000","last_up_change":"2024-04-28T12:47:58.139482+0000","last_in_change":"2024-04-28T12:47:37.513775+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-28T12:43:40.059873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-28T12:48:52.916511+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-28T12:48:53.528808+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-28T12:48:57.154563+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0afd8792-8351-40f5-ac93-536a58b5182a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6803","nonce":3844735961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6805","nonce":3844735961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6809","nonce":3844735961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":3844735961},{"type":"v1","addr":"172.21.15.106:6807","nonce":3844735961}]},"public_addr":"172.21.15.106:6803/3844735961","cluster_addr":"172.21.15.106:6805/3844735961","heartbeat_back_addr":"172.21.15.106:6809/3844735961","heartbeat_front_addr":"172.21.15.106:6807/3844735961","state":["exists","up"]},{"osd":1,"uuid":"d86a904e-10f6-402f-8107-f2fac6eead2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6801","nonce":3542000402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6803","nonce":3542000402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6807","nonce":3542000402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":3542000402},{"type":"v1","addr":"172.21.15.129:6805","nonce":3542000402}]},"public_addr":"172.21.15.129:6801/3542000402","cluster_addr":"172.21.15.129:6803/3542000402","heartbeat_back_addr":"172.21.15.129:6807/3542000402","heartbeat_front_addr":"172.21.15.129:6805/3542000402","state":["exists","up"]},{"osd":2,"uuid":"e9518ad8-2471-45eb-bf21-ab04fcd7274b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6801","nonce":1763026525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6803","nonce":1763026525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6807","nonce":1763026525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1763026525},{"type":"v1","addr":"172.21.15.171:6805","nonce":1763026525}]},"public_addr":"172.21.15.171:6801/1763026525","cluster_addr":"172.21.15.171:6803/1763026525","heartbeat_back_addr":"172.21.15.171:6807/1763026525","heartbeat_front_addr":"172.21.15.171:6805/1763026525","state":["exists","up"]},{"osd":3,"uuid":"4650110f-e49a-48fc-8ec3-8cc1b8ecdfb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6811","nonce":3527734792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6813","nonce":3527734792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6817","nonce":3527734792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":3527734792},{"type":"v1","addr":"172.21.15.106:6815","nonce":3527734792}]},"public_addr":"172.21.15.106:6811/3527734792","cluster_addr":"172.21.15.106:6813/3527734792","heartbeat_back_addr":"172.21.15.106:6817/3527734792","heartbeat_front_addr":"172.21.15.106:6815/3527734792","state":["exists","up"]},{"osd":4,"uuid":"505898bb-0925-4988-a527-d39888287e0e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6809","nonce":2253162285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6811","nonce":2253162285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6815","nonce":2253162285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":2253162285},{"type":"v1","addr":"172.21.15.129:6813","nonce":2253162285}]},"public_addr":"172.21.15.129:6809/2253162285","cluster_addr":"172.21.15.129:6811/2253162285","heartbeat_back_addr":"172.21.15.129:6815/2253162285","heartbeat_front_addr":"172.21.15.129:6813/2253162285","state":["exists","up"]},{"osd":5,"uuid":"b4836336-5952-4041-abaf-800c8058d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6809","nonce":1286181714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6811","nonce":1286181714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6815","nonce":1286181714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":1286181714},{"type":"v1","addr":"172.21.15.171:6813","nonce":1286181714}]},"public_addr":"172.21.15.171:6809/1286181714","cluster_addr":"172.21.15.171:6811/1286181714","heartbeat_back_addr":"172.21.15.171:6815/1286181714","heartbeat_front_addr":"172.21.15.171:6813/1286181714","state":["exists","up"]},{"osd":6,"uuid":"17da728b-3acb-48d3-9fda-4485aa4b3757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6819","nonce":1789189109}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6821","nonce":1789189109}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6825","nonce":1789189109}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":1789189109},{"type":"v1","addr":"172.21.15.106:6823","nonce":1789189109}]},"public_addr":"172.21.15.106:6819/1789189109","cluster_addr":"172.21.15.106:6821/1789189109","heartbeat_back_addr":"172.21.15.106:6825/1789189109","heartbeat_front_addr":"172.21.15.106:6823/1789189109","state":["exists","up"]},{"osd":7,"uuid":"9d679bae-c3a9-4c6b-a604-934ae327ca2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6817","nonce":3741780752}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6819","nonce":3741780752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6823","nonce":3741780752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":3741780752},{"type":"v1","addr":"172.21.15.129:6821","nonce":3741780752}]},"public_addr":"172.21.15.129:6817/3741780752","cluster_addr":"172.21.15.129:6819/3741780752","heartbeat_back_addr":"172.21.15.129:6823/3741780752","heartbeat_front_addr":"172.21.15.129:6821/3741780752","state":["exists","up"]},{"osd":8,"uuid":"9cf9f2b4-6bf1-4740-8170-456e3b3f858a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6817","nonce":1721308305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6819","nonce":1721308305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6823","nonce":1721308305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1721308305},{"type":"v1","addr":"172.21.15.171:6821","nonce":1721308305}]},"public_addr":"172.21.15.171:6817/1721308305","cluster_addr":"172.21.15.171:6819/1721308305","heartbeat_back_addr":"172.21.15.171:6823/1721308305","heartbeat_front_addr":"172.21.15.171:6821/1721308305","state":["exists","up"]},{"osd":9,"uuid":"cd4886f6-691b-4475-9180-11d5262b59a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6827","nonce":95511724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6829","nonce":95511724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6833","nonce":95511724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":95511724},{"type":"v1","addr":"172.21.15.106:6831","nonce":95511724}]},"public_addr":"172.21.15.106:6827/95511724","cluster_addr":"172.21.15.106:6829/95511724","heartbeat_back_addr":"172.21.15.106:6833/95511724","heartbeat_front_addr":"172.21.15.106:6831/95511724","state":["exists","up"]},{"osd":10,"uuid":"d2d8f70b-cc2e-43d1-b34f-d487b001e922","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6825","nonce":3282354257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6827","nonce":3282354257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6831","nonce":3282354257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":3282354257},{"type":"v1","addr":"172.21.15.129:6829","nonce":3282354257}]},"public_addr":"172.21.15.129:6825/3282354257","cluster_addr":"172.21.15.129:6827/3282354257","heartbeat_back_addr":"172.21.15.129:6831/3282354257","heartbeat_front_addr":"172.21.15.129:6829/3282354257","state":["exists","up"]},{"osd":11,"uuid":"dc45af10-78d3-4da4-b980-e7fde882529a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6825","nonce":2516393366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6827","nonce":2516393366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6831","nonce":2516393366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":2516393366},{"type":"v1","addr":"172.21.15.171:6829","nonce":2516393366}]},"public_addr":"172.21.15.171:6825/2516393366","cluster_addr":"172.21.15.171:6827/2516393366","heartbeat_back_addr":"172.21.15.171:6831/2516393366","heartbeat_front_addr":"172.21.15.171:6829/2516393366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:42:47.600361+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:12.746177+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:43:37.898317+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:10.465408+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:44:36.584805+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:02.599867+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:45:35.864979+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:02.554011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:46:28.484982+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:01.480716+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:29.289054+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-28T12:47:55.676604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.5","mappings":[{"from":3,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":8}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6801/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:6800/2253669851":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/3545890663":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/1369056801":"2024-04-29T12:40:55.904029+0000","172.21.15.106:0/938762297":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/1467729753":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6800/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2991131145":"2024-04-29T12:40:21.812705+0000","172.21.15.106:6801/102356115":"2024-04-29T12:40:21.812705+0000","172.21.15.106:0/2350345760":"2024-04-29T12:40:55.904029+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-28T13:06:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.507+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbd8c0a3cc0 msgr2=0x7fbd8c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.507+0000 7fbdad000640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbd8c0a3cc0 0x7fbd8c0a6180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbda8199e30 tx=0x7fbd90005f90 comp rx=0 tx=0).stop 2024-04-28T13:06:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.507+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8102750 msgr2=0x7fbda8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.507+0000 7fbdad000640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8102750 0x7fbda8198920 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800ebc0 tx=0x7fbd9800c6a0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.508+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 shutdown_connections 2024-04-28T13:06:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.508+0000 7fbdad000640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fbd8c0a3cc0 0x7fbd8c0a6180 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.508 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.508+0000 7fbdad000640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbda8104300 0x7fbda819fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.508+0000 7fbdad000640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbda8103940 0x7fbda8198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.508+0000 7fbdad000640 1 --2- 172.21.15.106:0/3012308678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbda8102750 0x7fbda8198920 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.508+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 >> 172.21.15.106:0/3012308678 conn(0x7fbda80fdcc0 msgr2=0x7fbda8106cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.508+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 shutdown_connections 2024-04-28T13:06:52.509 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.508+0000 7fbdad000640 1 -- 172.21.15.106:0/3012308678 wait complete. 2024-04-28T13:06:52.526 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-28T13:06:52.728 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.728+0000 7f8d6ea00640 1 -- 172.21.15.106:0/2172812017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d68106540 msgr2=0x7f8d6810cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.728+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/2172812017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d68106540 0x7f8d6810cdd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009ca0 tx=0x7f8d6002a8c0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.728+0000 7f8d6ea00640 1 -- 172.21.15.106:0/2172812017 shutdown_connections 2024-04-28T13:06:52.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.728+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/2172812017 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d68106540 0x7f8d6810cdd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.728+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/2172812017 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d68105b80 0x7f8d68106000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.728+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/2172812017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d68104990 0x7f8d68104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.728+0000 7f8d6ea00640 1 -- 172.21.15.106:0/2172812017 >> 172.21.15.106:0/2172812017 conn(0x7f8d680fff00 msgr2=0x7f8d68102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:52.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.728+0000 7f8d6ea00640 1 -- 172.21.15.106:0/2172812017 shutdown_connections 2024-04-28T13:06:52.729 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.728+0000 7f8d6ea00640 1 -- 172.21.15.106:0/2172812017 wait complete. 2024-04-28T13:06:52.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.729+0000 7f8d6ea00640 1 Processor -- start 2024-04-28T13:06:52.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.729+0000 7f8d6ea00640 1 -- start start 2024-04-28T13:06:52.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d68104990 0x7f8d6819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:52.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6ea00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d68105b80 0x7f8d6819b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:52.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d68106540 0x7f8d681a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:52.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6ea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d681104c0 con 0x7f8d68104990 2024-04-28T13:06:52.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6ea00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68110340 con 0x7f8d68105b80 2024-04-28T13:06:52.732 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6ea00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8d68110640 con 0x7f8d68106540 2024-04-28T13:06:52.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6d600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d68104990 0x7f8d6819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:52.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6d600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d68104990 0x7f8d6819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44448/0 (socket says 172.21.15.106:44448) 2024-04-28T13:06:52.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6d600640 1 -- 172.21.15.106:0/1852991643 learned_addr learned my addr 172.21.15.106:0/1852991643 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:52.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6cc00640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d68105b80 0x7f8d6819b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:52.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6e000640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d68106540 0x7f8d681a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:52.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6d600640 1 -- 172.21.15.106:0/1852991643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d68106540 msgr2=0x7f8d681a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6d600640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d68106540 0x7f8d681a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6d600640 1 -- 172.21.15.106:0/1852991643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d68105b80 msgr2=0x7f8d6819b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.733 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6d600640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d68105b80 0x7f8d6819b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.730+0000 7f8d6d600640 1 -- 172.21.15.106:0/1852991643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d681a2720 con 0x7f8d68104990 2024-04-28T13:06:52.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.731+0000 7f8d6d600640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d68104990 0x7f8d6819ab00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e9c0 tx=0x7f8d5c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:52.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.731+0000 7f8d66000640 1 -- 172.21.15.106:0/1852991643 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c00cd80 con 0x7f8d68104990 2024-04-28T13:06:52.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.731+0000 7f8d66000640 1 -- 172.21.15.106:0/1852991643 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d5c0035f0 con 0x7f8d68104990 2024-04-28T13:06:52.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.731+0000 7f8d66000640 1 -- 172.21.15.106:0/1852991643 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c010690 con 0x7f8d68104990 2024-04-28T13:06:52.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.731+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d681a2a10 con 0x7f8d68104990 2024-04-28T13:06:52.734 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.731+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d681a2f60 con 0x7f8d68104990 2024-04-28T13:06:52.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.732+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d6810bcb0 con 0x7f8d68104990 2024-04-28T13:06:52.735 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.734+0000 7f8d66000640 1 -- 172.21.15.106:0/1852991643 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8d5c0037a0 con 0x7f8d68104990 2024-04-28T13:06:52.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.736+0000 7f8d66000640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d500a3c40 0x7f8d500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:52.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.737+0000 7f8d66000640 1 -- 172.21.15.106:0/1852991643 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9216+0+0 (secure 0 0 0) 0x7f8d5c0d0d00 con 0x7f8d68104990 2024-04-28T13:06:52.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.737+0000 7f8d6cc00640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d500a3c40 0x7f8d500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:52.737 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.737+0000 7f8d6cc00640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d500a3c40 0x7f8d500a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8d6819c010 tx=0x7f8d58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:52.741 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.741+0000 7f8d66000640 1 -- 172.21.15.106:0/1852991643 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f8d5c09d170 con 0x7f8d68104990 2024-04-28T13:06:52.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:52 smithi171 ceph-mon[63127]: pgmap v860: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:52.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:52 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/2979255755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T13:06:52.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:52 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3115555963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T13:06:52.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:52 smithi129 ceph-mon[62418]: pgmap v860: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:52.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:52 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/2979255755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T13:06:52.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:52 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3115555963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T13:06:52.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:52 smithi106 ceph-mon[28081]: pgmap v860: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-28T13:06:52.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:52 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/2979255755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T13:06:52.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:52 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3115555963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-28T13:06:52.985 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.985+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8d6802dad0 con 0x7f8d68104990 2024-04-28T13:06:52.989 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.989+0000 7f8d66000640 1 -- 172.21.15.106:0/1852991643 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f8d5c09ca70 con 0x7f8d68104990 2024-04-28T13:06:52.989 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-28T13:06:52.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d500a3c40 msgr2=0x7f8d500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d500a3c40 0x7f8d500a6100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8d6819c010 tx=0x7f8d58005f90 comp rx=0 tx=0).stop 2024-04-28T13:06:52.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d68104990 msgr2=0x7f8d6819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:52.992 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d68104990 0x7f8d6819ab00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00e9c0 tx=0x7f8d5c00ee80 comp rx=0 tx=0).stop 2024-04-28T13:06:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 shutdown_connections 2024-04-28T13:06:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f8d500a3c40 0x7f8d500a6100 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8d6819c010 tx=0x7f8d58005f90 comp rx=0 tx=0).stop 2024-04-28T13:06:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8d68106540 0x7f8d681a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d68105b80 0x7f8d6819b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 --2- 172.21.15.106:0/1852991643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d68104990 0x7f8d6819ab00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:52.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.992+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 >> 172.21.15.106:0/1852991643 conn(0x7f8d680fff00 msgr2=0x7f8d68108e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:52.994 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.993+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 shutdown_connections 2024-04-28T13:06:52.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:52.994+0000 7f8d6ea00640 1 -- 172.21.15.106:0/1852991643 wait complete. 2024-04-28T13:06:53.024 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-28T13:06:53.212 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.212+0000 7f2d9e600640 1 -- 172.21.15.106:0/3029991528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d98104520 msgr2=0x7f2d9810adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:53.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.212+0000 7f2d9e600640 1 --2- 172.21.15.106:0/3029991528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d98104520 0x7f2d9810adb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009ca0 tx=0x7f2d9002a8c0 comp rx=0 tx=0).stop 2024-04-28T13:06:53.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.212+0000 7f2d9e600640 1 -- 172.21.15.106:0/3029991528 shutdown_connections 2024-04-28T13:06:53.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.212+0000 7f2d9e600640 1 --2- 172.21.15.106:0/3029991528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d98104520 0x7f2d9810adb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:53.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.212+0000 7f2d9e600640 1 --2- 172.21.15.106:0/3029991528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d98103b60 0x7f2d98103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:53.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.212+0000 7f2d9e600640 1 --2- 172.21.15.106:0/3029991528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d98102970 0x7f2d98102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:53.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.212+0000 7f2d9e600640 1 -- 172.21.15.106:0/3029991528 >> 172.21.15.106:0/3029991528 conn(0x7f2d980fdee0 msgr2=0x7f2d98100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:53.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.212+0000 7f2d9e600640 1 -- 172.21.15.106:0/3029991528 shutdown_connections 2024-04-28T13:06:53.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.212+0000 7f2d9e600640 1 -- 172.21.15.106:0/3029991528 wait complete. 2024-04-28T13:06:53.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.213+0000 7f2d9e600640 1 Processor -- start 2024-04-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.213+0000 7f2d9e600640 1 -- start start 2024-04-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.213+0000 7f2d9e600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d98102970 0x7f2d9819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.213+0000 7f2d9e600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d98103b60 0x7f2d9819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.213+0000 7f2d9e600640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d98104520 0x7f2d981a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d9e600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9810e4a0 con 0x7f2d98103b60 2024-04-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d9e600640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9810e320 con 0x7f2d98104520 2024-04-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d9e600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9810e620 con 0x7f2d98102970 2024-04-28T13:06:53.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d98102970 0x7f2d9819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d8fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d98103b60 0x7f2d9819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d98102970 0x7f2d9819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:45712/0 (socket says 172.21.15.106:45712) 2024-04-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97400640 1 -- 172.21.15.106:0/1445204718 learned_addr learned my addr 172.21.15.106:0/1445204718 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97e00640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d98104520 0x7f2d981a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97400640 1 -- 172.21.15.106:0/1445204718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d98104520 msgr2=0x7f2d981a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97400640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d98104520 0x7f2d981a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:53.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97400640 1 -- 172.21.15.106:0/1445204718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d98103b60 msgr2=0x7f2d9819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:53.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97400640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d98103b60 0x7f2d9819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:53.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97400640 1 -- 172.21.15.106:0/1445204718 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d981a2980 con 0x7f2d98102970 2024-04-28T13:06:53.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.214+0000 7f2d97400640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d98102970 0x7f2d9819ad60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800e9c0 tx=0x7f2d8800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:53.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.215+0000 7f2d94c00640 1 -- 172.21.15.106:0/1445204718 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800cd80 con 0x7f2d98102970 2024-04-28T13:06:53.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.215+0000 7f2d94c00640 1 -- 172.21.15.106:0/1445204718 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d880035f0 con 0x7f2d98102970 2024-04-28T13:06:53.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.215+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d981a2c70 con 0x7f2d98102970 2024-04-28T13:06:53.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.215+0000 7f2d94c00640 1 -- 172.21.15.106:0/1445204718 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88010690 con 0x7f2d98102970 2024-04-28T13:06:53.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.215+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a31c0 con 0x7f2d98102970 2024-04-28T13:06:53.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.216+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d981a3390 con 0x7f2d98102970 2024-04-28T13:06:53.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.218+0000 7f2d94c00640 1 -- 172.21.15.106:0/1445204718 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2d880037a0 con 0x7f2d98102970 2024-04-28T13:06:53.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.219+0000 7f2d94c00640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d7c0a3c40 0x7f2d7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:53.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.219+0000 7f2d94c00640 1 -- 172.21.15.106:0/1445204718 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9646+0+0 (secure 0 0 0) 0x7f2d880d0ff0 con 0x7f2d98102970 2024-04-28T13:06:53.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.220+0000 7f2d8fe00640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d7c0a3c40 0x7f2d7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:53.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.220+0000 7f2d8fe00640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d7c0a3c40 0x7f2d7c0a6100 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d9819c270 tx=0x7f2d80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:53.221 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.222+0000 7f2d94c00640 1 -- 172.21.15.106:0/1445204718 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7f2d8809d2f0 con 0x7f2d98102970 2024-04-28T13:06:53.328 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d[68273]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.329 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d[68273]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.329 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mds[68292]: -- 172.21.15.106:0/1611858076 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55655fe87a40 con 0x55655fe5ac00 2024-04-28T13:06:53.329 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d[68273]: starting mds.d at 2024-04-28T13:06:53.329 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g[68909]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.330 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g[68909]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.330 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mds[68913]: -- 172.21.15.106:0/2652396038 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fb1a81a40 con 0x563fb1a54800 2024-04-28T13:06:53.330 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g[68909]: starting mds.g at 2024-04-28T13:06:53.330 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:52 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a[28077]: 2024-04-28T13:06:52.986+0000 7f93b2000700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-28T13:06:53.331 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j[67928]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.331 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j[67928]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.331 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mds[67932]: -- 172.21.15.106:0/2790199265 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c8cfd7a40 con 0x562c8cfaac00 2024-04-28T13:06:53.331 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j[67928]: starting mds.j at 2024-04-28T13:06:53.331 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a[69267]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.331 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a[69267]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.332 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mds[69294]: -- 172.21.15.106:0/1851227291 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef08101a40 con 0x55ef080d5000 2024-04-28T13:06:53.332 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a[69267]: starting mds.a at 2024-04-28T13:06:53.409 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c[93099]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.409 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c[93099]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.409 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mds[93103]: -- 172.21.15.171:0/2914092884 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56189a251a40 con 0x56189a225000 2024-04-28T13:06:53.409 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c[93099]: starting mds.c at 2024-04-28T13:06:53.410 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f[92789]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.410 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f[92789]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.410 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mds[92793]: -- 172.21.15.171:0/2759466031 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56041d28da40 con 0x56041d260c00 2024-04-28T13:06:53.410 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f[92789]: starting mds.f at 2024-04-28T13:06:53.411 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i[93407]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.411 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i[93407]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.411 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mds[93411]: -- 172.21.15.171:0/1676216078 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5fcf5ba40 con 0x55a5fcf2f000 2024-04-28T13:06:53.411 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i[93407]: starting mds.i at 2024-04-28T13:06:53.412 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e[92256]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.412 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e[92256]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.412 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mds[92260]: -- 172.21.15.129:0/1160391497 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559bb32cfa40 con 0x559bb32a2c00 2024-04-28T13:06:53.412 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e[92256]: starting mds.e at 2024-04-28T13:06:53.412 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h[92566]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.412 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h[92566]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.413 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mds[92570]: -- 172.21.15.129:0/2722166140 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563087d83a40 con 0x563087d56c00 2024-04-28T13:06:53.413 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h[92566]: starting mds.h at 2024-04-28T13:06:53.413 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: ignoring --setuser ceph since I am not root 2024-04-28T13:06:53.413 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: ignoring --setgroup ceph since I am not root 2024-04-28T13:06:53.413 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mds[92877]: -- 172.21.15.129:0/3284931563 <== mon.2 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a3325e7a40 con 0x55a3325bb000 2024-04-28T13:06:53.413 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: starting mds.b at 2024-04-28T13:06:53.465 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.465+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2d981a3560 con 0x7f2d98102970 2024-04-28T13:06:53.718 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3012308678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T13:06:53.718 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1852991643' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-28T13:06:53.718 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mon[28081]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-28T13:06:53.718 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mon[28081]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-28T13:06:53.719 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1852991643' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-28T13:06:53.719 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mon[28081]: osdmap e94: 12 total, 12 up, 12 in 2024-04-28T13:06:53.719 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:53 smithi106 ceph-mon[28081]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-28T13:06:53.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3012308678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T13:06:53.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1852991643' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-28T13:06:53.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mon[63127]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-28T13:06:53.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mon[63127]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-28T13:06:53.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1852991643' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-28T13:06:53.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mon[63127]: osdmap e94: 12 total, 12 up, 12 in 2024-04-28T13:06:53.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:53 smithi171 ceph-mon[63127]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-28T13:06:53.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3012308678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-28T13:06:53.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1852991643' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-28T13:06:53.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mon[62418]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-28T13:06:53.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mon[62418]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-28T13:06:53.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1852991643' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-28T13:06:53.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mon[62418]: osdmap e94: 12 total, 12 up, 12 in 2024-04-28T13:06:53.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:53 smithi129 ceph-mon[62418]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-28T13:06:53.995 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.995+0000 7f2d94c00640 1 -- 172.21.15.106:0/1445204718 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f2d8809cbf0 con 0x7f2d98102970 2024-04-28T13:06:53.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.999+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d7c0a3c40 msgr2=0x7f2d7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:53.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.999+0000 7f2d9e600640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d7c0a3c40 0x7f2d7c0a6100 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d9819c270 tx=0x7f2d80005f90 comp rx=0 tx=0).stop 2024-04-28T13:06:53.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.999+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d98102970 msgr2=0x7f2d9819ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:53.999 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:53.999+0000 7f2d9e600640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d98102970 0x7f2d9819ad60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800e9c0 tx=0x7f2d8800ee80 comp rx=0 tx=0).stop 2024-04-28T13:06:54.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.004+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 shutdown_connections 2024-04-28T13:06:54.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.004+0000 7f2d9e600640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7f2d7c0a3c40 0x7f2d7c0a6100 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.004+0000 7f2d9e600640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2d98103b60 0x7f2d9819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.004+0000 7f2d9e600640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d98104520 0x7f2d981a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.005+0000 7f2d9e600640 1 --2- 172.21.15.106:0/1445204718 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2d98102970 0x7f2d9819ad60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.005+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 >> 172.21.15.106:0/1445204718 conn(0x7f2d980fdee0 msgr2=0x7f2d98109320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:54.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.005+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 shutdown_connections 2024-04-28T13:06:54.005 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.005+0000 7f2d9e600640 1 -- 172.21.15.106:0/1445204718 wait complete. 2024-04-28T13:06:54.025 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-28T13:06:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.210+0000 7fd72a200640 1 -- 172.21.15.106:0/2748911483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7241011e0 msgr2=0x7fd724103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.210+0000 7fd72a200640 1 --2- 172.21.15.106:0/2748911483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7241011e0 0x7fd724103610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd714009900 tx=0x7fd71402a3f0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.211+0000 7fd72a200640 1 -- 172.21.15.106:0/2748911483 shutdown_connections 2024-04-28T13:06:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.211+0000 7fd72a200640 1 --2- 172.21.15.106:0/2748911483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd724106500 0x7fd724106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.211+0000 7fd72a200640 1 --2- 172.21.15.106:0/2748911483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724103b50 0x7fd724105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.211+0000 7fd72a200640 1 --2- 172.21.15.106:0/2748911483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7241011e0 0x7fd724103610 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.211+0000 7fd72a200640 1 -- 172.21.15.106:0/2748911483 >> 172.21.15.106:0/2748911483 conn(0x7fd7240fcde0 msgr2=0x7fd7240ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.211+0000 7fd72a200640 1 -- 172.21.15.106:0/2748911483 shutdown_connections 2024-04-28T13:06:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.211+0000 7fd72a200640 1 -- 172.21.15.106:0/2748911483 wait complete. 2024-04-28T13:06:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd72a200640 1 Processor -- start 2024-04-28T13:06:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd72a200640 1 -- start start 2024-04-28T13:06:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd72a200640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7241011e0 0x7fd72419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd72a200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724103b50 0x7fd72419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd72a200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd724106500 0x7fd7241a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd72a200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd724114b10 con 0x7fd724106500 2024-04-28T13:06:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd72a200640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd724114990 con 0x7fd7241011e0 2024-04-28T13:06:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd72a200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd724114c90 con 0x7fd724103b50 2024-04-28T13:06:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd723e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd724106500 0x7fd7241a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd723400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7241011e0 0x7fd72419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd723400640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7241011e0 0x7fd72419f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:41422/0 (socket says 172.21.15.106:41422) 2024-04-28T13:06:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.212+0000 7fd723400640 1 -- 172.21.15.106:0/3668238043 learned_addr learned my addr 172.21.15.106:0/3668238043 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd722a00640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724103b50 0x7fd72419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd723400640 1 -- 172.21.15.106:0/3668238043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724103b50 msgr2=0x7fd72419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd723400640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724103b50 0x7fd72419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.215 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd723400640 1 -- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd724106500 msgr2=0x7fd7241a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:54.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd723400640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd724106500 0x7fd7241a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:54.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd723400640 1 -- 172.21.15.106:0/3668238043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7241a6df0 con 0x7fd7241011e0 2024-04-28T13:06:54.216 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd723e00640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd724106500 0x7fd7241a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:06:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd723400640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7241011e0 0x7fd72419f1d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd714005ab0 tx=0x7fd714003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd71be00640 1 -- 172.21.15.106:0/3668238043 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71403f070 con 0x7fd7241011e0 2024-04-28T13:06:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7241a7080 con 0x7fd7241011e0 2024-04-28T13:06:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd71be00640 1 -- 172.21.15.106:0/3668238043 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd714031c50 con 0x7fd7241011e0 2024-04-28T13:06:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.213+0000 7fd71be00640 1 -- 172.21.15.106:0/3668238043 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714035030 con 0x7fd7241011e0 2024-04-28T13:06:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.214+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7241a75c0 con 0x7fd7241011e0 2024-04-28T13:06:54.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.215+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7240fe8d0 con 0x7fd7241011e0 2024-04-28T13:06:54.218 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.217+0000 7fd71be00640 1 -- 172.21.15.106:0/3668238043 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd714042070 con 0x7fd7241011e0 2024-04-28T13:06:54.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.218+0000 7fd71be00640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7080a3c40 0x7fd7080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:54.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.218+0000 7fd71be00640 1 -- 172.21.15.106:0/3668238043 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9646+0+0 (secure 0 0 0) 0x7fd7140f1970 con 0x7fd7241011e0 2024-04-28T13:06:54.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.218+0000 7fd722a00640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7080a3c40 0x7fd7080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:54.219 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.219+0000 7fd722a00640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7080a3c40 0x7fd7080a6100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd7241a06e0 tx=0x7fd710005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:54.220 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.220+0000 7fd71be00640 1 -- 172.21.15.106:0/3668238043 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd7140bd5c0 con 0x7fd7241011e0 2024-04-28T13:06:54.425 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:54.425+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd72402dad0 con 0x7fd7241011e0 2024-04-28T13:06:54.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: pgmap v861: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-28T13:06:54.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/1445204718' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-28T13:06:54.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-28T13:06:54.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-28T13:06:54.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-28T13:06:54.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: Cluster is now healthy 2024-04-28T13:06:54.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-28T13:06:54.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.129:6834/3805124738,v1:172.21.15.129:6835/3805124738] up:boot 2024-04-28T13:06:54.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.171:6834/803215595,v1:172.21.15.171:6835/803215595] up:boot 2024-04-28T13:06:54.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.129:6836/2678529219,v1:172.21.15.129:6837/2678529219] up:boot 2024-04-28T13:06:54.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.129:6838/3254763067,v1:172.21.15.129:6839/3254763067] up:boot 2024-04-28T13:06:54.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.171:6836/515550592,v1:172.21.15.171:6838/515550592] up:boot 2024-04-28T13:06:54.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.171:6837/371919658,v1:172.21.15.171:6839/371919658] up:boot 2024-04-28T13:06:54.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6834/249624172,v1:172.21.15.106:6835/249624172] up:boot 2024-04-28T13:06:54.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6836/1280434145,v1:172.21.15.106:6837/1280434145] up:boot 2024-04-28T13:06:54.751 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6838/2299138402,v1:172.21.15.106:6839/2299138402] up:boot 2024-04-28T13:06:54.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: mds.? [v2:172.21.15.106:6840/2798912263,v1:172.21.15.106:6841/2798912263] up:boot 2024-04-28T13:06:54.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: fsmap 12 up:standby 2024-04-28T13:06:54.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-28T13:06:54.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-28T13:06:54.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-28T13:06:54.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-28T13:06:54.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-28T13:06:54.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-28T13:06:54.752 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-28T13:06:54.753 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-28T13:06:54.753 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-28T13:06:54.753 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:54 smithi171 ceph-mon[63127]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-28T13:06:54.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: pgmap v861: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-28T13:06:54.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/1445204718' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-28T13:06:54.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-28T13:06:54.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-28T13:06:54.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-28T13:06:54.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: Cluster is now healthy 2024-04-28T13:06:54.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-28T13:06:54.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.129:6834/3805124738,v1:172.21.15.129:6835/3805124738] up:boot 2024-04-28T13:06:54.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.171:6834/803215595,v1:172.21.15.171:6835/803215595] up:boot 2024-04-28T13:06:54.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.129:6836/2678529219,v1:172.21.15.129:6837/2678529219] up:boot 2024-04-28T13:06:54.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.129:6838/3254763067,v1:172.21.15.129:6839/3254763067] up:boot 2024-04-28T13:06:54.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.171:6836/515550592,v1:172.21.15.171:6838/515550592] up:boot 2024-04-28T13:06:54.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.171:6837/371919658,v1:172.21.15.171:6839/371919658] up:boot 2024-04-28T13:06:54.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6834/249624172,v1:172.21.15.106:6835/249624172] up:boot 2024-04-28T13:06:54.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6836/1280434145,v1:172.21.15.106:6837/1280434145] up:boot 2024-04-28T13:06:54.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6838/2299138402,v1:172.21.15.106:6839/2299138402] up:boot 2024-04-28T13:06:54.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: mds.? [v2:172.21.15.106:6840/2798912263,v1:172.21.15.106:6841/2798912263] up:boot 2024-04-28T13:06:54.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: fsmap 12 up:standby 2024-04-28T13:06:54.769 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-28T13:06:54.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-28T13:06:54.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-28T13:06:54.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-28T13:06:54.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-28T13:06:54.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-28T13:06:54.770 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-28T13:06:54.771 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-28T13:06:54.771 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-28T13:06:54.771 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:54 smithi129 ceph-mon[62418]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-28T13:06:54.830 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: pgmap v861: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-28T13:06:54.830 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/1445204718' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-28T13:06:54.830 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-28T13:06:54.830 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-28T13:06:54.830 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-28T13:06:54.831 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: Cluster is now healthy 2024-04-28T13:06:54.831 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-28T13:06:54.831 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.129:6834/3805124738,v1:172.21.15.129:6835/3805124738] up:boot 2024-04-28T13:06:54.831 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.171:6834/803215595,v1:172.21.15.171:6835/803215595] up:boot 2024-04-28T13:06:54.831 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.129:6836/2678529219,v1:172.21.15.129:6837/2678529219] up:boot 2024-04-28T13:06:54.831 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.129:6838/3254763067,v1:172.21.15.129:6839/3254763067] up:boot 2024-04-28T13:06:54.831 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.171:6836/515550592,v1:172.21.15.171:6838/515550592] up:boot 2024-04-28T13:06:54.831 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.171:6837/371919658,v1:172.21.15.171:6839/371919658] up:boot 2024-04-28T13:06:54.832 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6834/249624172,v1:172.21.15.106:6835/249624172] up:boot 2024-04-28T13:06:54.832 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6836/1280434145,v1:172.21.15.106:6837/1280434145] up:boot 2024-04-28T13:06:54.832 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6838/2299138402,v1:172.21.15.106:6839/2299138402] up:boot 2024-04-28T13:06:54.832 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: mds.? [v2:172.21.15.106:6840/2798912263,v1:172.21.15.106:6841/2798912263] up:boot 2024-04-28T13:06:54.832 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: fsmap 12 up:standby 2024-04-28T13:06:54.832 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-28T13:06:54.832 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-28T13:06:54.832 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-28T13:06:54.833 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-28T13:06:54.833 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-28T13:06:54.833 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-28T13:06:54.833 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-28T13:06:54.833 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-28T13:06:54.833 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-28T13:06:54.833 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:54 smithi106 ceph-mon[28081]: from='mgr.14152 172.21.15.106:0/4285697667' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-28T13:06:55.438 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.438+0000 7fd71be00640 1 -- 172.21.15.106:0/3668238043 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7fd7140c18c0 con 0x7fd7241011e0 2024-04-28T13:06:55.438 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2024-04-28T13:06:55.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7080a3c40 msgr2=0x7fd7080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:55.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7080a3c40 0x7fd7080a6100 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd7241a06e0 tx=0x7fd710005f90 comp rx=0 tx=0).stop 2024-04-28T13:06:55.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7241011e0 msgr2=0x7fd72419f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:55.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7241011e0 0x7fd72419f1d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd714005ab0 tx=0x7fd714003b90 comp rx=0 tx=0).stop 2024-04-28T13:06:55.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 shutdown_connections 2024-04-28T13:06:55.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd7080a3c40 0x7fd7080a6100 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:55.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd724106500 0x7fd7241a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:55.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd724103b50 0x7fd72419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:55.444 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 --2- 172.21.15.106:0/3668238043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd7241011e0 0x7fd72419f1d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.443+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 >> 172.21.15.106:0/3668238043 conn(0x7fd7240fcde0 msgr2=0x7fd7241045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.444+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 shutdown_connections 2024-04-28T13:06:55.445 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.444+0000 7fd72a200640 1 -- 172.21.15.106:0/3668238043 wait complete. 2024-04-28T13:06:55.466 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-28T13:06:55.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.670+0000 7ff2dde00640 1 -- 172.21.15.106:0/3232915502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d8076970 msgr2=0x7ff2d8074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:55.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.670+0000 7ff2dde00640 1 --2- 172.21.15.106:0/3232915502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d8076970 0x7ff2d8074dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009930 tx=0x7ff2c802a420 comp rx=0 tx=0).stop 2024-04-28T13:06:55.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.670+0000 7ff2dde00640 1 -- 172.21.15.106:0/3232915502 shutdown_connections 2024-04-28T13:06:55.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.670+0000 7ff2dde00640 1 --2- 172.21.15.106:0/3232915502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2d810ee90 0x7ff2d8111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:55.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.670+0000 7ff2dde00640 1 --2- 172.21.15.106:0/3232915502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2d8075390 0x7ff2d8075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:55.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.670+0000 7ff2dde00640 1 --2- 172.21.15.106:0/3232915502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d8076970 0x7ff2d8074dc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:55.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.670+0000 7ff2dde00640 1 -- 172.21.15.106:0/3232915502 >> 172.21.15.106:0/3232915502 conn(0x7ff2d80fde30 msgr2=0x7ff2d8100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:55.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.670+0000 7ff2dde00640 1 -- 172.21.15.106:0/3232915502 shutdown_connections 2024-04-28T13:06:55.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.670+0000 7ff2dde00640 1 -- 172.21.15.106:0/3232915502 wait complete. 2024-04-28T13:06:55.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.671+0000 7ff2dde00640 1 Processor -- start 2024-04-28T13:06:55.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.671+0000 7ff2dde00640 1 -- start start 2024-04-28T13:06:55.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dde00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2d8075390 0x7ff2d819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:55.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dde00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2d8076970 0x7ff2d819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:55.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d810ee90 0x7ff2d81a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:55.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dde00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8114970 con 0x7ff2d810ee90 2024-04-28T13:06:55.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dde00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d81147f0 con 0x7ff2d8076970 2024-04-28T13:06:55.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dde00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d8114af0 con 0x7ff2d8075390 2024-04-28T13:06:55.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dd400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d810ee90 0x7ff2d81a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:55.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2d7e00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2d8076970 0x7ff2d819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:55.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dd400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d810ee90 0x7ff2d81a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44578/0 (socket says 172.21.15.106:44578) 2024-04-28T13:06:55.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dd400640 1 -- 172.21.15.106:0/395024809 learned_addr learned my addr 172.21.15.106:0/395024809 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:55.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2dca00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2d8075390 0x7ff2d819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:55.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2d7e00640 1 -- 172.21.15.106:0/395024809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2d8075390 msgr2=0x7ff2d819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:55.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2d7e00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2d8075390 0x7ff2d819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:55.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2d7e00640 1 -- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d810ee90 msgr2=0x7ff2d81a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:55.675 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.672+0000 7ff2d7e00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d810ee90 0x7ff2d81a6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:55.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.673+0000 7ff2d7e00640 1 -- 172.21.15.106:0/395024809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d81a6d00 con 0x7ff2d8076970 2024-04-28T13:06:55.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.673+0000 7ff2dd400640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d810ee90 0x7ff2d81a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-28T13:06:55.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.673+0000 7ff2d7e00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2d8076970 0x7ff2d819f620 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400e9f0 tx=0x7ff2c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:55.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.673+0000 7ff2d5600640 1 -- 172.21.15.106:0/395024809 <== mon.2 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400cd80 con 0x7ff2d8076970 2024-04-28T13:06:55.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.673+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d81a6ff0 con 0x7ff2d8076970 2024-04-28T13:06:55.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.673+0000 7ff2d5600640 1 -- 172.21.15.106:0/395024809 <== mon.2 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c40035f0 con 0x7ff2d8076970 2024-04-28T13:06:55.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.673+0000 7ff2d5600640 1 -- 172.21.15.106:0/395024809 <== mon.2 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4010690 con 0x7ff2d8076970 2024-04-28T13:06:55.676 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.673+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d81a7500 con 0x7ff2d8076970 2024-04-28T13:06:55.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.674+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d8105060 con 0x7ff2d8076970 2024-04-28T13:06:55.677 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.677+0000 7ff2d5600640 1 -- 172.21.15.106:0/395024809 <== mon.2 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7ff2c40037a0 con 0x7ff2d8076970 2024-04-28T13:06:55.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.678+0000 7ff2d5600640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff2bc0a3c40 0x7ff2bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:55.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.678+0000 7ff2d5600640 1 -- 172.21.15.106:0/395024809 <== mon.2 v2:172.21.15.129:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9142+0+0 (secure 0 0 0) 0x7ff2c40d0c90 con 0x7ff2d8076970 2024-04-28T13:06:55.678 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.678+0000 7ff2dca00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff2bc0a3c40 0x7ff2bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:55.679 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.679+0000 7ff2dca00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff2bc0a3c40 0x7ff2bc0a6100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8006010 tx=0x7ff2c80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:55.680 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.680+0000 7ff2d5600640 1 -- 172.21.15.106:0/395024809 <== mon.2 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7ff2c409d1b0 con 0x7ff2d8076970 2024-04-28T13:06:55.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:55 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3668238043' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:55.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:55 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:55.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:55 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3668238043' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:55.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:55 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:55.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:55 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3668238043' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:55.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:55 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:55.894 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:55.894+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff2d802dad0 con 0x7ff2d8076970 2024-04-28T13:06:56.443 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.442+0000 7ff2d5600640 1 -- 172.21.15.106:0/395024809 <== mon.2 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7ff2c409cab0 con 0x7ff2d8076970 2024-04-28T13:06:56.443 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2024-04-28T13:06:56.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.450+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff2bc0a3c40 msgr2=0x7ff2bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:56.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.450+0000 7ff2dde00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff2bc0a3c40 0x7ff2bc0a6100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8006010 tx=0x7ff2c80314a0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.450+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2d8076970 msgr2=0x7ff2d819f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:56.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.450+0000 7ff2dde00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2d8076970 0x7ff2d819f620 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400e9f0 tx=0x7ff2c400eeb0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.451+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 shutdown_connections 2024-04-28T13:06:56.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.451+0000 7ff2dde00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7ff2bc0a3c40 0x7ff2bc0a6100 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.451+0000 7ff2dde00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2d810ee90 0x7ff2d81a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.451+0000 7ff2dde00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2d8076970 0x7ff2d819f620 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.451+0000 7ff2dde00640 1 --2- 172.21.15.106:0/395024809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff2d8075390 0x7ff2d819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.451 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.451+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 >> 172.21.15.106:0/395024809 conn(0x7ff2d80fde30 msgr2=0x7ff2d8102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:56.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.451+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 shutdown_connections 2024-04-28T13:06:56.452 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.451+0000 7ff2dde00640 1 -- 172.21.15.106:0/395024809 wait complete. 2024-04-28T13:06:56.474 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-28T13:06:56.664 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.664+0000 7fd05cc00640 1 -- 172.21.15.106:0/2799691897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0581024e0 msgr2=0x7fd058102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:56.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.664+0000 7fd05cc00640 1 --2- 172.21.15.106:0/2799691897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0581024e0 0x7fd058102960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd050009900 tx=0x7fd05002a410 comp rx=0 tx=0).stop 2024-04-28T13:06:56.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.664+0000 7fd05cc00640 1 -- 172.21.15.106:0/2799691897 shutdown_connections 2024-04-28T13:06:56.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.664+0000 7fd05cc00640 1 --2- 172.21.15.106:0/2799691897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0581024e0 0x7fd058102960 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.664+0000 7fd05cc00640 1 --2- 172.21.15.106:0/2799691897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd058101b40 0x7fd058101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.664+0000 7fd05cc00640 1 --2- 172.21.15.106:0/2799691897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0581011e0 0x7fd058101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.664+0000 7fd05cc00640 1 -- 172.21.15.106:0/2799691897 >> 172.21.15.106:0/2799691897 conn(0x7fd0580fcde0 msgr2=0x7fd0580ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:56.665 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.664+0000 7fd05cc00640 1 -- 172.21.15.106:0/2799691897 shutdown_connections 2024-04-28T13:06:56.666 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.664+0000 7fd05cc00640 1 -- 172.21.15.106:0/2799691897 wait complete. 2024-04-28T13:06:56.667 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.665+0000 7fd05cc00640 1 Processor -- start 2024-04-28T13:06:56.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.665+0000 7fd05cc00640 1 -- start start 2024-04-28T13:06:56.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.665+0000 7fd05cc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0581011e0 0x7fd05819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:56.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd05cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd058101b40 0x7fd05819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:56.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd05cc00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0581024e0 0x7fd0581a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:56.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd05cc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581106e0 con 0x7fd058101b40 2024-04-28T13:06:56.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd05cc00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd058110560 con 0x7fd0581024e0 2024-04-28T13:06:56.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd05cc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd058110860 con 0x7fd0581011e0 2024-04-28T13:06:56.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd055e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0581011e0 0x7fd05819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:56.668 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd056800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0581024e0 0x7fd0581a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:56.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd055e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0581011e0 0x7fd05819ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.106:45742/0 (socket says 172.21.15.106:45742) 2024-04-28T13:06:56.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd055e00640 1 -- 172.21.15.106:0/3318783397 learned_addr learned my addr 172.21.15.106:0/3318783397 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-28T13:06:56.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd056800640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0581024e0 0x7fd0581a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.106:41454/0 (socket says 172.21.15.106:41454) 2024-04-28T13:06:56.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd055400640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd058101b40 0x7fd05819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:56.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd055e00640 1 -- 172.21.15.106:0/3318783397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0581024e0 msgr2=0x7fd0581a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:56.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd055e00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0581024e0 0x7fd0581a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd055e00640 1 -- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd058101b40 msgr2=0x7fd05819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:56.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd055e00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd058101b40 0x7fd05819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:56.669 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.666+0000 7fd055e00640 1 -- 172.21.15.106:0/3318783397 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0581a2970 con 0x7fd0581011e0 2024-04-28T13:06:56.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.667+0000 7fd055e00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0581011e0 0x7fd05819ad50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e9f0 tx=0x7fd04800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:56.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.667+0000 7fd04ea00640 1 -- 172.21.15.106:0/3318783397 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04800cd80 con 0x7fd0581011e0 2024-04-28T13:06:56.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.667+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0581a2c60 con 0x7fd0581011e0 2024-04-28T13:06:56.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.667+0000 7fd04ea00640 1 -- 172.21.15.106:0/3318783397 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0480054d0 con 0x7fd0581011e0 2024-04-28T13:06:56.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.667+0000 7fd04ea00640 1 -- 172.21.15.106:0/3318783397 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048010690 con 0x7fd0581011e0 2024-04-28T13:06:56.670 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.667+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0581a31b0 con 0x7fd0581011e0 2024-04-28T13:06:56.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.668+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05810c0c0 con 0x7fd0581011e0 2024-04-28T13:06:56.671 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.670+0000 7fd04ea00640 1 -- 172.21.15.106:0/3318783397 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fd048004e30 con 0x7fd0581011e0 2024-04-28T13:06:56.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.672+0000 7fd04ea00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0380a3c40 0x7fd0380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-28T13:06:56.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.672+0000 7fd04ea00640 1 -- 172.21.15.106:0/3318783397 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8496+0+0 (secure 0 0 0) 0x7fd0480d0ce0 con 0x7fd0581011e0 2024-04-28T13:06:56.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.672+0000 7fd055400640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0380a3c40 0x7fd0380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-28T13:06:56.672 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.673+0000 7fd055400640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0380a3c40 0x7fd0380a6100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd05819c260 tx=0x7fd040009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-28T13:06:56.674 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.674+0000 7fd04ea00640 1 -- 172.21.15.106:0/3318783397 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188494 (secure 0 0 0) 0x7fd048054060 con 0x7fd0581011e0 2024-04-28T13:06:56.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:56 smithi171 ceph-mon[63127]: pgmap v863: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-28T13:06:56.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:56 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-28T13:06:56.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:56 smithi171 ceph-mon[63127]: osdmap e95: 12 total, 12 up, 12 in 2024-04-28T13:06:56.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:56 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/395024809' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:56.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:56 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:56.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:56 smithi129 ceph-mon[62418]: pgmap v863: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-28T13:06:56.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:56 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-28T13:06:56.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:56 smithi129 ceph-mon[62418]: osdmap e95: 12 total, 12 up, 12 in 2024-04-28T13:06:56.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:56 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/395024809' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:56.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:56 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:56.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:56 smithi106 ceph-mon[28081]: pgmap v863: 129 pgs: 129 active+clean; 640 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-28T13:06:56.828 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:56 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-28T13:06:56.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:56 smithi106 ceph-mon[28081]: osdmap e95: 12 total, 12 up, 12 in 2024-04-28T13:06:56.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:56 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/395024809' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:56.829 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:56 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:56.881 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:56.880+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd05802dad0 con 0x7fd0581011e0 2024-04-28T13:06:57.450 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.449+0000 7fd04ea00640 1 -- 172.21.15.106:0/3318783397 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) ==== 178+0+0 (secure 0 0 0) 0x7fd04809d420 con 0x7fd0581011e0 2024-04-28T13:06:57.450 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data_ec' removed 2024-04-28T13:06:57.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.454+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0380a3c40 msgr2=0x7fd0380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:57.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.454+0000 7fd05cc00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0380a3c40 0x7fd0380a6100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd05819c260 tx=0x7fd040009500 comp rx=0 tx=0).stop 2024-04-28T13:06:57.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.454+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0581011e0 msgr2=0x7fd05819ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-28T13:06:57.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.454+0000 7fd05cc00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0581011e0 0x7fd05819ad50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e9f0 tx=0x7fd04800edf0 comp rx=0 tx=0).stop 2024-04-28T13:06:57.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.455+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 shutdown_connections 2024-04-28T13:06:57.455 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.455+0000 7fd05cc00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:6800/1556498064,v1:172.21.15.106:6801/1556498064] conn(0x7fd0380a3c40 0x7fd0380a6100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:57.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.455+0000 7fd05cc00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd058101b40 0x7fd05819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:57.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.455+0000 7fd05cc00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd0581024e0 0x7fd0581a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:57.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.455+0000 7fd05cc00640 1 --2- 172.21.15.106:0/3318783397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd0581011e0 0x7fd05819ad50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-28T13:06:57.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.455+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 >> 172.21.15.106:0/3318783397 conn(0x7fd0580fcde0 msgr2=0x7fd0580fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-28T13:06:57.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.455+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 shutdown_connections 2024-04-28T13:06:57.456 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-28T13:06:57.456+0000 7fd05cc00640 1 -- 172.21.15.106:0/3318783397 wait complete. 2024-04-28T13:06:57.477 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-28T13:06:57.549 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-28T13:06:57.581 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-28T13:06:57.613 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-28T13:06:57.613 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-28T13:06:57.670 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-28T13:06:57.687 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-28T13:06:57.705 INFO:tasks.cephadm:Stopping all daemons... 2024-04-28T13:06:57.705 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-28T13:06:57.705 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.a 2024-04-28T13:06:57.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:57 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-28T13:06:57.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:57 smithi171 ceph-mon[63127]: osdmap e96: 12 total, 12 up, 12 in 2024-04-28T13:06:57.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:57 smithi171 ceph-mon[63127]: pgmap v866: 33 pgs: 33 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T13:06:57.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:57 smithi171 ceph-mon[63127]: from='client.? 172.21.15.106:0/3318783397' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:57.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:06:57 smithi171 ceph-mon[63127]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:57.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:57 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-28T13:06:57.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:57 smithi129 ceph-mon[62418]: osdmap e96: 12 total, 12 up, 12 in 2024-04-28T13:06:57.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:57 smithi129 ceph-mon[62418]: pgmap v866: 33 pgs: 33 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T13:06:57.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:57 smithi129 ceph-mon[62418]: from='client.? 172.21.15.106:0/3318783397' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:57.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:57 smithi129 ceph-mon[62418]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:57.787 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:57 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-28T13:06:57.787 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:57 smithi106 ceph-mon[28081]: osdmap e96: 12 total, 12 up, 12 in 2024-04-28T13:06:57.787 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:57 smithi106 ceph-mon[28081]: pgmap v866: 33 pgs: 33 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-28T13:06:57.787 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:57 smithi106 ceph-mon[28081]: from='client.? 172.21.15.106:0/3318783397' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:57.787 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:57 smithi106 ceph-mon[28081]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-28T13:06:58.079 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:57 smithi106 systemd[1]: Stopping Ceph mon.a for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:06:58.489 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a[28077]: 2024-04-28T13:06:58.225+0000 7f93b8e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:06:58.490 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a[28077]: 2024-04-28T13:06:58.225+0000 7f93b8e00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-28T13:06:58.490 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:58 smithi106 podman[85383]: 2024-04-28 13:06:58.463267994 +0000 UTC m=+0.375591712 container stop d6af359f5189fcbd9d4732bbc68c339405cfa8ca36afb66efb3c3f83ece8ccfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-28T13:06:58.490 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:58 smithi106 podman[85383]: 2024-04-28 13:06:58.490209041 +0000 UTC m=+0.402532705 container died d6af359f5189fcbd9d4732bbc68c339405cfa8ca36afb66efb3c3f83ece8ccfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-28T13:06:59.329 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:59 smithi106 podman[85383]: 2024-04-28 13:06:59.028480309 +0000 UTC m=+0.940804010 container cleanup d6af359f5189fcbd9d4732bbc68c339405cfa8ca36afb66efb3c3f83ece8ccfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-28T13:06:59.329 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:59 smithi106 bash[85383]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a 2024-04-28T13:06:59.330 INFO:journalctl@ceph.mon.a.smithi106.stdout:Apr 28 13:06:59 smithi106 podman[85397]: 2024-04-28 13:06:59.255393051 +0000 UTC m=+0.772054071 container remove d6af359f5189fcbd9d4732bbc68c339405cfa8ca36afb66efb3c3f83ece8ccfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:06:59.794 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.a.service' 2024-04-28T13:06:59.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:06:59.938 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-28T13:06:59.939 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-28T13:06:59.939 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.b 2024-04-28T13:07:00.267 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:06:59 smithi129 systemd[1]: Stopping Ceph mon.b for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:00.767 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:07:00 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-b[62414]: 2024-04-28T13:07:00.285+0000 7efca2620700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:00.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:07:00 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-b[62414]: 2024-04-28T13:07:00.285+0000 7efca2620700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-28T13:07:00.768 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:07:00 smithi129 podman[97210]: 2024-04-28 13:07:00.470446338 +0000 UTC m=+0.340746805 container died ff947229bf24b5d74c586529eb5fd666b0cd9b996178fbc9b484ce46ac6854e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True) 2024-04-28T13:07:01.268 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:07:01 smithi129 podman[97210]: 2024-04-28 13:07:01.033202788 +0000 UTC m=+0.903503266 container cleanup ff947229bf24b5d74c586529eb5fd666b0cd9b996178fbc9b484ce46ac6854e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-28T13:07:01.268 INFO:journalctl@ceph.mon.b.smithi129.stdout:Apr 28 13:07:01 smithi129 bash[97210]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-b 2024-04-28T13:07:01.459 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.b.service' 2024-04-28T13:07:01.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:01.508 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-28T13:07:01.508 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-28T13:07:01.508 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.c 2024-04-28T13:07:01.816 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:07:01 smithi171 systemd[1]: Stopping Ceph mon.c for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:02.201 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:07:01 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-c[63108]: 2024-04-28T13:07:01.815+0000 7f84767fc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:02.201 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:07:01 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-c[63108]: 2024-04-28T13:07:01.815+0000 7f84767fc700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-28T13:07:02.201 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:07:01 smithi171 podman[97744]: 2024-04-28 13:07:01.887693693 +0000 UTC m=+0.184107252 container died 997c6c7c6a4e2562565c4e2c49c67f47c8b593b083edb33827c83e6bbe042046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-28T13:07:02.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:07:02 smithi171 podman[97744]: 2024-04-28 13:07:02.393043074 +0000 UTC m=+0.689456624 container cleanup 997c6c7c6a4e2562565c4e2c49c67f47c8b593b083edb33827c83e6bbe042046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-28T13:07:02.500 INFO:journalctl@ceph.mon.c.smithi171.stdout:Apr 28 13:07:02 smithi171 bash[97744]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mon-c 2024-04-28T13:07:02.819 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mon.c.service' 2024-04-28T13:07:02.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:02.864 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-28T13:07:02.864 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-28T13:07:02.864 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.x 2024-04-28T13:07:03.328 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 13:07:02 smithi106 systemd[1]: Stopping Ceph mgr.x for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:03.829 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 13:07:03 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T13:07:03.423+0000 7f2bc0a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:03.829 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 13:07:03 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x[28310]: 2024-04-28T13:07:03.423+0000 7f2bc0a00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-28T13:07:03.829 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 13:07:03 smithi106 podman[85504]: 2024-04-28 13:07:03.516682512 +0000 UTC m=+0.225020850 container died 489a4dc1e7dce79d543fd6675215a3930fbcc2a6e4c815e4a6baf58c91de9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-28T13:07:04.092 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 13:07:04 smithi106 podman[85504]: 2024-04-28 13:07:04.02644237 +0000 UTC m=+0.734780745 container cleanup 489a4dc1e7dce79d543fd6675215a3930fbcc2a6e4c815e4a6baf58c91de9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-04-28T13:07:04.093 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 13:07:04 smithi106 bash[85504]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x 2024-04-28T13:07:04.578 INFO:journalctl@ceph.mgr.x.smithi106.stdout:Apr 28 13:07:04 smithi106 podman[85517]: 2024-04-28 13:07:04.219746162 +0000 UTC m=+0.691761888 container remove 489a4dc1e7dce79d543fd6675215a3930fbcc2a6e4c815e4a6baf58c91de9ea6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc612ffb9c1d5bc27536bc917a705262a36d4387, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-04-28T13:07:04.706 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-28T13:07:04.810 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:04.810 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-28T13:07:04.811 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-28T13:07:04.811 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.y 2024-04-28T13:07:05.150 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 13:07:04 smithi129 systemd[1]: Stopping Ceph mgr.y for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:05.480 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 13:07:05 smithi129 podman[97334]: 2024-04-28 13:07:05.151068022 +0000 UTC m=+0.143422623 container died 2f3649a5e5d28d47cae563bc662209ac401082082a19c0ff0734f07a448e49a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:07:05.767 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 13:07:05 smithi129 podman[97334]: 2024-04-28 13:07:05.580762554 +0000 UTC m=+0.573117135 container cleanup 2f3649a5e5d28d47cae563bc662209ac401082082a19c0ff0734f07a448e49a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-28T13:07:05.767 INFO:journalctl@ceph.mgr.y.smithi129.stdout:Apr 28 13:07:05 smithi129 bash[97334]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-y 2024-04-28T13:07:06.049 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-28T13:07:06.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:06.082 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-28T13:07:06.082 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-28T13:07:06.083 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.z 2024-04-28T13:07:06.435 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 13:07:06 smithi171 systemd[1]: Stopping Ceph mgr.z for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:06.731 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 13:07:06 smithi171 podman[97869]: 2024-04-28 13:07:06.435337367 +0000 UTC m=+0.159702477 container died 8c848c386a46302567c95392d5c01125b8731d8cdbe3c9421743323534fdcdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:07:06.999 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 13:07:06 smithi171 podman[97869]: 2024-04-28 13:07:06.815285786 +0000 UTC m=+0.539650907 container cleanup 8c848c386a46302567c95392d5c01125b8731d8cdbe3c9421743323534fdcdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-28T13:07:07.000 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Apr 28 13:07:06 smithi171 bash[97869]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mgr-z 2024-04-28T13:07:07.237 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-28T13:07:07.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:07.271 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-28T13:07:07.271 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-28T13:07:07.271 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.0 2024-04-28T13:07:07.805 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:07 smithi106 systemd[1]: Stopping Ceph osd.0 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:08.078 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:07 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0[42459]: 2024-04-28T13:07:07.804+0000 7f71bac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:08.079 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:07 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0[42459]: 2024-04-28T13:07:07.804+0000 7f71bac00700 -1 osd.0 97 *** Got signal Terminated *** 2024-04-28T13:07:08.079 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:07 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0[42459]: 2024-04-28T13:07:07.804+0000 7f71bac00700 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:07:13.537 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:13 smithi106 podman[85625]: 2024-04-28 13:07:13.199646579 +0000 UTC m=+5.537368093 container died 08c2d668e8f2785f96d4086fe3a36b8a6ca7f047bca48c7f509de1ecaa4f9d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:07:13.828 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:13 smithi106 podman[85625]: 2024-04-28 13:07:13.664873416 +0000 UTC m=+6.002594960 container cleanup 08c2d668e8f2785f96d4086fe3a36b8a6ca7f047bca48c7f509de1ecaa4f9d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-28T13:07:13.829 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:13 smithi106 bash[85625]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0 2024-04-28T13:07:14.328 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:13 smithi106 podman[85643]: 2024-04-28 13:07:13.900118746 +0000 UTC m=+0.687228808 container remove 08c2d668e8f2785f96d4086fe3a36b8a6ca7f047bca48c7f509de1ecaa4f9d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:07:15.040 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:14 smithi106 podman[85737]: 2024-04-28 13:07:14.649934499 +0000 UTC m=+0.035767933 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:07:15.041 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:14 smithi106 podman[85737]: 2024-04-28 13:07:14.75092452 +0000 UTC m=+0.136757995 container create 2ad8ff57c80d3a8bf2c092c038ec5af5704e9d8f4234190f4ee2b3dc3bee2155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-28T13:07:15.329 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:15 smithi106 podman[85737]: 2024-04-28 13:07:15.040244687 +0000 UTC m=+0.426078192 container init 2ad8ff57c80d3a8bf2c092c038ec5af5704e9d8f4234190f4ee2b3dc3bee2155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-28T13:07:15.329 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:15 smithi106 podman[85737]: 2024-04-28 13:07:15.051595431 +0000 UTC m=+0.437428887 container start 2ad8ff57c80d3a8bf2c092c038ec5af5704e9d8f4234190f4ee2b3dc3bee2155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-28T13:07:15.329 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:15 smithi106 podman[85737]: 2024-04-28 13:07:15.145627999 +0000 UTC m=+0.531461503 container attach 2ad8ff57c80d3a8bf2c092c038ec5af5704e9d8f4234190f4ee2b3dc3bee2155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-28T13:07:15.828 INFO:journalctl@ceph.osd.0.smithi106.stdout:Apr 28 13:07:15 smithi106 podman[85737]: 2024-04-28 13:07:15.504320644 +0000 UTC m=+0.890154125 container died 2ad8ff57c80d3a8bf2c092c038ec5af5704e9d8f4234190f4ee2b3dc3bee2155 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:07:16.285 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.0.service' 2024-04-28T13:07:16.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:16.374 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-28T13:07:16.374 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-28T13:07:16.374 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.1 2024-04-28T13:07:16.702 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:16 smithi129 systemd[1]: Stopping Ceph osd.1 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:17.017 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:16 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1[67013]: 2024-04-28T13:07:16.701+0000 7fd9ebba3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:17.017 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:16 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1[67013]: 2024-04-28T13:07:16.701+0000 7fd9ebba3700 -1 osd.1 97 *** Got signal Terminated *** 2024-04-28T13:07:17.018 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:16 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1[67013]: 2024-04-28T13:07:16.701+0000 7fd9ebba3700 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:07:22.328 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:22 smithi129 podman[97461]: 2024-04-28 13:07:22.035775238 +0000 UTC m=+5.457665147 container died e3eaf3eede8fc00c5404d92622ed9c8f553aa169af937bde22c7db6d91154520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:07:22.745 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:22 smithi129 podman[97461]: 2024-04-28 13:07:22.427418861 +0000 UTC m=+5.849308767 container cleanup e3eaf3eede8fc00c5404d92622ed9c8f553aa169af937bde22c7db6d91154520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:07:22.745 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:22 smithi129 bash[97461]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1 2024-04-28T13:07:23.017 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:22 smithi129 podman[97478]: 2024-04-28 13:07:22.745491777 +0000 UTC m=+0.702058360 container remove e3eaf3eede8fc00c5404d92622ed9c8f553aa169af937bde22c7db6d91154520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-28T13:07:23.373 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:23 smithi129 podman[97570]: 2024-04-28 13:07:23.005263276 +0000 UTC m=+0.018601558 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:07:23.373 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:23 smithi129 podman[97570]: 2024-04-28 13:07:23.129987969 +0000 UTC m=+0.143326253 container create 5628d7f083cf83a8e32d7262dd288a4d7a7138213b6f11404e1e0de1f9b09518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-28T13:07:23.374 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:23 smithi129 podman[97570]: 2024-04-28 13:07:23.374165129 +0000 UTC m=+0.387503412 container init 5628d7f083cf83a8e32d7262dd288a4d7a7138213b6f11404e1e0de1f9b09518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-28T13:07:23.670 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:23 smithi129 podman[97570]: 2024-04-28 13:07:23.377305644 +0000 UTC m=+0.390643928 container start 5628d7f083cf83a8e32d7262dd288a4d7a7138213b6f11404e1e0de1f9b09518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-28T13:07:23.670 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:23 smithi129 podman[97570]: 2024-04-28 13:07:23.457531346 +0000 UTC m=+0.470869639 container attach 5628d7f083cf83a8e32d7262dd288a4d7a7138213b6f11404e1e0de1f9b09518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD) 2024-04-28T13:07:24.017 INFO:journalctl@ceph.osd.1.smithi129.stdout:Apr 28 13:07:23 smithi129 podman[97570]: 2024-04-28 13:07:23.67109514 +0000 UTC m=+0.684433429 container died 5628d7f083cf83a8e32d7262dd288a4d7a7138213b6f11404e1e0de1f9b09518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-28T13:07:24.405 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.1.service' 2024-04-28T13:07:24.448 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:24.448 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-28T13:07:24.448 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-28T13:07:24.448 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.2 2024-04-28T13:07:24.750 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:24 smithi171 systemd[1]: Stopping Ceph osd.2 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:25.010 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:24 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2[68288]: 2024-04-28T13:07:24.844+0000 7f2c91750700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:25.010 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:24 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2[68288]: 2024-04-28T13:07:24.844+0000 7f2c91750700 -1 osd.2 97 *** Got signal Terminated *** 2024-04-28T13:07:25.011 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:24 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2[68288]: 2024-04-28T13:07:24.844+0000 7f2c91750700 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:07:30.712 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:30 smithi171 podman[97998]: 2024-04-28 13:07:30.366647656 +0000 UTC m=+5.725936345 container died 3220d67fb743222bf8338b1f9f8abc60ce5f06cb32b358d760bc33f9d441c0a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T13:07:30.999 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:30 smithi171 podman[97998]: 2024-04-28 13:07:30.803627572 +0000 UTC m=+6.162916269 container cleanup 3220d67fb743222bf8338b1f9f8abc60ce5f06cb32b358d760bc33f9d441c0a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:07:31.000 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:30 smithi171 bash[97998]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2 2024-04-28T13:07:31.476 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:31 smithi171 podman[98027]: 2024-04-28 13:07:31.030613495 +0000 UTC m=+0.657541043 container remove 3220d67fb743222bf8338b1f9f8abc60ce5f06cb32b358d760bc33f9d441c0a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424) 2024-04-28T13:07:31.749 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:31 smithi171 podman[98117]: 2024-04-28 13:07:31.380109735 +0000 UTC m=+0.018542732 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:07:31.750 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:31 smithi171 podman[98117]: 2024-04-28 13:07:31.498031991 +0000 UTC m=+0.136464974 container create 64fb9618d503deb6861dafe4fecdce98ced65586d35a0659d3efada6cd4cf742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-28T13:07:32.048 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:31 smithi171 podman[98117]: 2024-04-28 13:07:31.75094986 +0000 UTC m=+0.389382859 container init 64fb9618d503deb6861dafe4fecdce98ced65586d35a0659d3efada6cd4cf742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-04-28T13:07:32.048 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:31 smithi171 podman[98117]: 2024-04-28 13:07:31.754018223 +0000 UTC m=+0.392451212 container start 64fb9618d503deb6861dafe4fecdce98ced65586d35a0659d3efada6cd4cf742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-28T13:07:32.048 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:31 smithi171 podman[98117]: 2024-04-28 13:07:31.850852393 +0000 UTC m=+0.489285378 container attach 64fb9618d503deb6861dafe4fecdce98ced65586d35a0659d3efada6cd4cf742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:07:32.412 INFO:journalctl@ceph.osd.2.smithi171.stdout:Apr 28 13:07:32 smithi171 podman[98117]: 2024-04-28 13:07:32.048523588 +0000 UTC m=+0.686956563 container died 64fb9618d503deb6861dafe4fecdce98ced65586d35a0659d3efada6cd4cf742 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-28T13:07:32.727 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.2.service' 2024-04-28T13:07:32.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:32.766 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-28T13:07:32.767 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-28T13:07:32.767 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.3 2024-04-28T13:07:33.329 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:32 smithi106 systemd[1]: Stopping Ceph osd.3 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:33.828 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:33 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3[49246]: 2024-04-28T13:07:33.334+0000 7f8ed5c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:33.829 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:33 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3[49246]: 2024-04-28T13:07:33.334+0000 7f8ed5c00700 -1 osd.3 97 *** Got signal Terminated *** 2024-04-28T13:07:33.829 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:33 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3[49246]: 2024-04-28T13:07:33.334+0000 7f8ed5c00700 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:07:34.249 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:33 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:33.860+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:34.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:34 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:34.417+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:35.078 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:34 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3[49246]: 2024-04-28T13:07:34.633+0000 7f8ed1400700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.377138+0000 front 2024-04-28T13:07:08.377163+0000 (oldest deadline 2024-04-28T13:07:34.177710+0000) 2024-04-28T13:07:35.249 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:34 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:34.848+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:35.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:34 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:34.880+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:35.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:35 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:35.079+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:35.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:35 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:35.383+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:36.051 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:35 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3[49246]: 2024-04-28T13:07:35.654+0000 7f8ed1400700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.377138+0000 front 2024-04-28T13:07:08.377163+0000 (oldest deadline 2024-04-28T13:07:34.177710+0000) 2024-04-28T13:07:36.249 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:35 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:35.830+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:36.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:35 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:35.928+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:36.328 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:36 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:36.050+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:36.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:36 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:36.383+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:37.031 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:36 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3[49246]: 2024-04-28T13:07:36.636+0000 7f8ed1400700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.377138+0000 front 2024-04-28T13:07:08.377163+0000 (oldest deadline 2024-04-28T13:07:34.177710+0000) 2024-04-28T13:07:37.249 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:36 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:36.858+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:37.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:36 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:36.911+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:37.328 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:37 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:37.030+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:37.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:37 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:37.336+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:38.059 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:37 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3[49246]: 2024-04-28T13:07:37.626+0000 7f8ed1400700 -1 osd.3 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.377138+0000 front 2024-04-28T13:07:08.377163+0000 (oldest deadline 2024-04-28T13:07:34.177710+0000) 2024-04-28T13:07:38.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:37.947+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:38.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:37.824+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:38.328 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:38 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:38.058+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:38.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:38 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:38.292+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:39.175 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:38 smithi106 podman[85854]: 2024-04-28 13:07:38.891258578 +0000 UTC m=+5.703550276 container died e5912e529bb5da9be1fa92881967cab4f1c7604f7da89fe62c3065951b0e4b1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.license=GPLv2) 2024-04-28T13:07:39.175 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:39 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:39.071+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:39.249 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:38.872+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:39.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:38.998+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:39.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:39 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:39.253+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:39.579 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:39 smithi106 podman[85854]: 2024-04-28 13:07:39.320314582 +0000 UTC m=+6.132606256 container cleanup e5912e529bb5da9be1fa92881967cab4f1c7604f7da89fe62c3065951b0e4b1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:07:39.579 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:39 smithi106 bash[85854]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3 2024-04-28T13:07:40.078 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:39 smithi106 podman[85874]: 2024-04-28 13:07:39.58017743 +0000 UTC m=+0.674790478 container remove e5912e529bb5da9be1fa92881967cab4f1c7604f7da89fe62c3065951b0e4b1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-28T13:07:40.249 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:39.823+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:40.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:39.996+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:40.395 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:40 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:40.086+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:40.693 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:40 smithi106 podman[85966]: 2024-04-28 13:07:40.301427307 +0000 UTC m=+0.033748212 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:07:40.694 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:40 smithi106 podman[85966]: 2024-04-28 13:07:40.422778701 +0000 UTC m=+0.155099643 container create d3924759c0cf78950bc38e2c548475a1c7a4edee13410c4e7069d7b2d5e439ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:07:40.694 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:40 smithi106 podman[85966]: 2024-04-28 13:07:40.693333225 +0000 UTC m=+0.425654177 container init d3924759c0cf78950bc38e2c548475a1c7a4edee13410c4e7069d7b2d5e439ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-28T13:07:40.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:40 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:40.295+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:41.079 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:40 smithi106 podman[85966]: 2024-04-28 13:07:40.704727529 +0000 UTC m=+0.437048468 container start d3924759c0cf78950bc38e2c548475a1c7a4edee13410c4e7069d7b2d5e439ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-28T13:07:41.079 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:40 smithi106 podman[85966]: 2024-04-28 13:07:40.783661063 +0000 UTC m=+0.515982052 container attach d3924759c0cf78950bc38e2c548475a1c7a4edee13410c4e7069d7b2d5e439ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-28T13:07:41.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:40 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:40.949+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:41.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:40 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:40.830+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:41.492 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:41 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:41.079+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:41.493 INFO:journalctl@ceph.osd.3.smithi106.stdout:Apr 28 13:07:41 smithi106 podman[85966]: 2024-04-28 13:07:41.151707031 +0000 UTC m=+0.884027946 container died d3924759c0cf78950bc38e2c548475a1c7a4edee13410c4e7069d7b2d5e439ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-28T13:07:41.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:41 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:41.267+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:41.858 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.3.service' 2024-04-28T13:07:41.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:41.952 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-28T13:07:41.953 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-28T13:07:41.953 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.4 2024-04-28T13:07:42.110 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:41.825+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:42.111 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:41.977+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:42.248 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:42 smithi129 systemd[1]: Stopping Ceph osd.4 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:42.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:42 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:42.247+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:42.517 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:42 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4[73514]: 2024-04-28T13:07:42.264+0000 7f0ae1112700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:42.518 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:42 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4[73514]: 2024-04-28T13:07:42.264+0000 7f0ae1112700 -1 osd.4 97 *** Got signal Terminated *** 2024-04-28T13:07:42.518 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:42 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4[73514]: 2024-04-28T13:07:42.264+0000 7f0ae1112700 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:07:42.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:42 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:42.101+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:43.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:42.998+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:43.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:42.874+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:43.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:42.874+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:43.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:43 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:43.079+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:43.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:43 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:43.079+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:43.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:43 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:43.286+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:44.072 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:43 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:43.675+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:44.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:44.038+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:44.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:43.885+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:44.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:43.885+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:44.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:44 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:44.070+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:44.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:44 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:44.070+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:44.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:44 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:44.247+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:44.930 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:44 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4[73514]: 2024-04-28T13:07:44.631+0000 7f0ad96fe700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:19.048002+0000 front 2024-04-28T13:07:19.048128+0000 (oldest deadline 2024-04-28T13:07:43.748149+0000) 2024-04-28T13:07:45.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:44 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:44.704+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:45.215 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:44 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:44.929+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:45.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:44.990+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:45.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:44.914+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:45.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:44.914+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:45.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:44.911+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:45.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:45 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:45.214+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:45.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:45 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:45.086+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:45.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:45 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:45.086+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:45.889 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:45 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4[73514]: 2024-04-28T13:07:45.618+0000 7f0ad96fe700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:19.048002+0000 front 2024-04-28T13:07:19.048128+0000 (oldest deadline 2024-04-28T13:07:43.748149+0000) 2024-04-28T13:07:46.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:45 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:45.680+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:46.249 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:45 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:45.951+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:46.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:45 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:45.951+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:46.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:45 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:45.953+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:46.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:46 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:46.030+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:46.259 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:45 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:45.889+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:46.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:46 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:46.259+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:46.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:46 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:46.259+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:46.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:46 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:46.126+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:46.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:46 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:46.126+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:47.017 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:46 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4[73514]: 2024-04-28T13:07:46.637+0000 7f0ad96fe700 -1 osd.4 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:19.048002+0000 front 2024-04-28T13:07:19.048128+0000 (oldest deadline 2024-04-28T13:07:43.748149+0000) 2024-04-28T13:07:47.018 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:46 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:46.853+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:47.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:46 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:46.697+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:47.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:46 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:46.976+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:47.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:47 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:47.032+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:47.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:46 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:46.906+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:47.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:46 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:46.906+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:47.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:47.107+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:47.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:47.107+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:47.641 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:47 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:47.300+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:47.641 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:47 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:47.300+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:47.915 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:47 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:47.890+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:47.915 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:47 smithi129 podman[97690]: 2024-04-28 13:07:47.642298947 +0000 UTC m=+5.491323344 container died 5c896d29b2ef6accf98ca7c01d02e4a2c9af1299bf8082b1cf3eb8ce023df4e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-28T13:07:48.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:47.712+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:48.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:47 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:47.947+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:48.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:48 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:48.005+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:48.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:47 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:47.951+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:48.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:47 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:47.951+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:48.267 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:48 smithi129 podman[97690]: 2024-04-28 13:07:48.023583081 +0000 UTC m=+5.872607526 container cleanup 5c896d29b2ef6accf98ca7c01d02e4a2c9af1299bf8082b1cf3eb8ce023df4e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-28T13:07:48.267 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:48 smithi129 bash[97690]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4 2024-04-28T13:07:48.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:48 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:48.153+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:48.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:48 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:48.153+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:48.654 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:48 smithi129 podman[97708]: 2024-04-28 13:07:48.308265416 +0000 UTC m=+0.659608687 container remove 5c896d29b2ef6accf98ca7c01d02e4a2c9af1299bf8082b1cf3eb8ce023df4e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-28T13:07:48.654 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:48 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:48.269+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:48.654 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:48 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:48.269+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:48.936 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:48 smithi129 podman[97798]: 2024-04-28 13:07:48.557673332 +0000 UTC m=+0.017909902 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:07:48.936 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:48 smithi129 podman[97798]: 2024-04-28 13:07:48.684070927 +0000 UTC m=+0.144307497 container create 38275211222de712429740f115bbc5a8885bbfe43e8aa6763236c7615b083410 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-28T13:07:48.936 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:48 smithi129 podman[97798]: 2024-04-28 13:07:48.936599664 +0000 UTC m=+0.396836235 container init 38275211222de712429740f115bbc5a8885bbfe43e8aa6763236c7615b083410 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-28T13:07:49.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:48 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:48.714+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:49.230 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:48 smithi129 podman[97798]: 2024-04-28 13:07:48.939732356 +0000 UTC m=+0.399968928 container start 38275211222de712429740f115bbc5a8885bbfe43e8aa6763236c7615b083410 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-28T13:07:49.231 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:49 smithi129 podman[97798]: 2024-04-28 13:07:49.028421153 +0000 UTC m=+0.488657735 container attach 38275211222de712429740f115bbc5a8885bbfe43e8aa6763236c7615b083410 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-04-28T13:07:49.231 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:48 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:48.939+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:49.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:48 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:48.926+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:49.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:48 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:48.971+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:49.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:48 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:48.983+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:49.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:48 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:48.983+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:49.517 INFO:journalctl@ceph.osd.4.smithi129.stdout:Apr 28 13:07:49 smithi129 podman[97798]: 2024-04-28 13:07:49.230908928 +0000 UTC m=+0.691145506 container died 38275211222de712429740f115bbc5a8885bbfe43e8aa6763236c7615b083410 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-28T13:07:49.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:49 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:49.267+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:49.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:49 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:49.267+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:49.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:49 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:49.122+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:49.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:49 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:49.122+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:49.855 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.4.service' 2024-04-28T13:07:49.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:49.894 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-28T13:07:49.894 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-28T13:07:49.894 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.5 2024-04-28T13:07:50.081 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:49 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:49.753+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:50.216 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:49 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:49.888+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:50.216 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:49 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:49.933+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:50.217 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:49 smithi171 systemd[1]: Stopping Ceph osd.5 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:50.217 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:49 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:49.986+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:50.217 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:49 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:49.986+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:50.217 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:50 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:50.216+0000 7f4455b74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:50.217 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:50 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:50.216+0000 7f4455b74700 -1 osd.5 97 *** Got signal Terminated *** 2024-04-28T13:07:50.217 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:50 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:50.216+0000 7f4455b74700 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:07:50.229 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:49 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:49.984+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:50.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:50 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:50.229+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:50.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:50 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:50.229+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:50.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:50 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:50.150+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:50.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:50 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:50.150+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:51.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:50 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:50.798+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:51.233 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:50 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:50.969+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:51.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:50 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:50.909+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:51.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:50 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:50.887+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:51.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:51.002+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:51.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:51.002+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:51.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:51.002+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.672463+0000 front 2024-04-28T13:07:29.672495+0000 (oldest deadline 2024-04-28T13:07:50.673270+0000) 2024-04-28T13:07:51.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:51 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:51.233+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:51.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:51 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:51.233+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:51.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:51 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:51.135+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:51.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:51 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:51.135+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:51.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:51 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:51.135+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:07:52.098 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:51 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:51.834+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:52.249 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:51.991+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:52.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:51.991+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:52.250 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:51.991+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.672463+0000 front 2024-04-28T13:07:29.672495+0000 (oldest deadline 2024-04-28T13:07:50.673270+0000) 2024-04-28T13:07:52.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:51.900+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:52.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:51.900+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:07:52.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:51.885+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:52.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:51 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:51.885+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:07:52.267 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:52 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:52.249+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:52.267 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:52 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:52.249+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:52.267 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:52 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:52.008+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:52.268 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:52 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:52.008+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:07:52.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:52 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:52.098+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:52.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:52 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:52.098+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:52.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:52 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:52.098+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:07:53.065 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:52 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:52.806+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:53.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:52 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:52.873+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:53.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:52 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:52.873+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:07:53.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:52 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:52.940+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:53.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:52 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:52.940+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:07:53.251 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:53.012+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:53.251 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:53.012+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:53.251 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:53.012+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.672463+0000 front 2024-04-28T13:07:29.672495+0000 (oldest deadline 2024-04-28T13:07:50.673270+0000) 2024-04-28T13:07:53.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:53.065+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:53.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:53.065+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:53.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:53.065+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:07:53.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:53.227+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:53.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:53.227+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:53.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:53.227+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:07:53.518 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:53.021+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:53.518 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:53 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:53.021+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:07:54.050 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:53 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:53.795+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:54.151 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:53.828+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:54.151 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:53.828+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:07:54.151 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:53.964+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:54.151 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:53 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:53.964+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:07:54.152 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:54.021+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:54.152 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:54.021+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:54.152 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:54.021+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.672463+0000 front 2024-04-28T13:07:29.672495+0000 (oldest deadline 2024-04-28T13:07:50.673270+0000) 2024-04-28T13:07:54.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:54.050+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:54.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:54.050+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:54.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:54.050+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:07:54.517 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:54 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:54.062+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:54.517 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:54 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:54.062+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:07:54.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:54 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:54.264+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:54.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:54 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:54.264+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:54.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:54 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:54.264+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:07:55.067 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:54.786+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:55.068 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:54.786+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.952408+0000 front 2024-04-28T13:07:28.952483+0000 (oldest deadline 2024-04-28T13:07:54.153760+0000) 2024-04-28T13:07:55.164 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:54.992+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:08.967516+0000 front 2024-04-28T13:07:08.967571+0000 (oldest deadline 2024-04-28T13:07:33.568077+0000) 2024-04-28T13:07:55.164 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:54.992+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:16.469250+0000 front 2024-04-28T13:07:16.469273+0000 (oldest deadline 2024-04-28T13:07:42.269970+0000) 2024-04-28T13:07:55.165 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5[74937]: 2024-04-28T13:07:54.992+0000 7f444e160700 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.672463+0000 front 2024-04-28T13:07:29.672495+0000 (oldest deadline 2024-04-28T13:07:50.673270+0000) 2024-04-28T13:07:55.165 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:54.932+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:55.165 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:54.932+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:07:55.165 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:54.789+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:55.166 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:54 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:54.789+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:07:55.328 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:55 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:55.067+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:55.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:55 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:55.067+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:55.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:55 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:55.067+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:07:55.517 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:55 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:55.108+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:55.518 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:55 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:55.108+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:07:55.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:55 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:55.295+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:55.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:55 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:55.295+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:55.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:55 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:55.295+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:07:55.750 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:55 smithi171 podman[98222]: 2024-04-28 13:07:55.647197129 +0000 UTC m=+5.561334763 container died 0bc73af7e636fb884bfec52536ee9135e58fd9227d4de37164faf942cb788ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:07:56.057 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:55 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:55.777+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:56.057 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:55 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:55.777+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:07:56.057 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:55 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:55.890+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:56.057 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:55 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:55.890+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:07:56.079 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:55 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:55.778+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:56.079 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:55 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:55.778+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.952408+0000 front 2024-04-28T13:07:28.952483+0000 (oldest deadline 2024-04-28T13:07:54.153760+0000) 2024-04-28T13:07:56.402 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:56 smithi171 podman[98222]: 2024-04-28 13:07:56.057530612 +0000 UTC m=+5.971668243 container cleanup 0bc73af7e636fb884bfec52536ee9135e58fd9227d4de37164faf942cb788ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:07:56.402 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:56 smithi171 bash[98222]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5 2024-04-28T13:07:56.402 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:56 smithi171 podman[98248]: 2024-04-28 13:07:56.300560709 +0000 UTC m=+0.655692566 container remove 0bc73af7e636fb884bfec52536ee9135e58fd9227d4de37164faf942cb788ffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-28T13:07:56.517 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:56 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:56.129+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:56.518 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:56 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:56.129+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:07:56.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:56 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:56.334+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:56.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:56 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:56.334+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:56.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:56 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:56.334+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:07:56.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:56 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:56.115+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:56.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:56 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:56.115+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:56.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:56 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:56.115+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:07:56.678 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:56 smithi171 podman[98339]: 2024-04-28 13:07:56.528378906 +0000 UTC m=+0.018363912 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:07:56.678 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:56 smithi171 podman[98339]: 2024-04-28 13:07:56.678956191 +0000 UTC m=+0.168941219 container create 0e3ce139c6b19e44461698e68bb7457a3ef7828f8668130c77c39d0014bdaf2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-28T13:07:56.940 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:56 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:56.810+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:56.941 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:56 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:56.810+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:07:56.941 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:56 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:56.844+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:56.941 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:56 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:56.844+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:07:57.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:56 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:56.798+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:57.079 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:56 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:56.798+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.952408+0000 front 2024-04-28T13:07:28.952483+0000 (oldest deadline 2024-04-28T13:07:54.153760+0000) 2024-04-28T13:07:57.238 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:56 smithi171 podman[98339]: 2024-04-28 13:07:56.941259808 +0000 UTC m=+0.431244801 container init 0e3ce139c6b19e44461698e68bb7457a3ef7828f8668130c77c39d0014bdaf2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:07:57.239 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:56 smithi171 podman[98339]: 2024-04-28 13:07:56.944536827 +0000 UTC m=+0.434521818 container start 0e3ce139c6b19e44461698e68bb7457a3ef7828f8668130c77c39d0014bdaf2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-28T13:07:57.239 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:57 smithi171 podman[98339]: 2024-04-28 13:07:57.016448541 +0000 UTC m=+0.506433539 container attach 0e3ce139c6b19e44461698e68bb7457a3ef7828f8668130c77c39d0014bdaf2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:07:57.239 INFO:journalctl@ceph.osd.5.smithi171.stdout:Apr 28 13:07:57 smithi171 podman[98339]: 2024-04-28 13:07:57.238808915 +0000 UTC m=+0.728793899 container died 0e3ce139c6b19e44461698e68bb7457a3ef7828f8668130c77c39d0014bdaf2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5) 2024-04-28T13:07:57.517 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:57 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:57.096+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:57.518 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:57 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:57.096+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:07:57.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:57 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:57.338+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:57.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:57 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:57.338+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:57.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:57 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:57.338+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:07:57.578 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:57 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:57.093+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:57.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:57 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:57.093+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:57.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:57 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:57.093+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:07:57.874 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.5.service' 2024-04-28T13:07:57.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:07:57.911 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-28T13:07:57.911 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-28T13:07:57.911 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.6 2024-04-28T13:07:58.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:57 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:57.819+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:58.079 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:57 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:57.819+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.952408+0000 front 2024-04-28T13:07:28.952483+0000 (oldest deadline 2024-04-28T13:07:54.153760+0000) 2024-04-28T13:07:58.079 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:58 smithi106 systemd[1]: Stopping Ceph osd.6 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:07:58.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:57 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:57.802+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:58.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:57 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:57.802+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:07:58.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:57 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:57.817+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:58.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:57 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:57.817+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:07:58.499 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:58.079+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:58.499 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:58.079+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:58.499 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:58.079+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:07:58.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:58 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:58.316+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:58.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:58 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:58.316+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:58.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:58 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:58.316+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:07:58.518 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:58 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:58.123+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:58.518 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:58 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:58.123+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:07:58.798 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:58.498+0000 7fa861600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:07:58.798 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:58.498+0000 7fa861600700 -1 osd.6 97 *** Got signal Terminated *** 2024-04-28T13:07:58.798 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:58.498+0000 7fa861600700 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:07:58.798 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:58.796+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:07:58.798 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:58.796+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.952408+0000 front 2024-04-28T13:07:28.952483+0000 (oldest deadline 2024-04-28T13:07:54.153760+0000) 2024-04-28T13:07:59.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:58 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:58.796+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:07:59.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:58 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:58.796+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:07:59.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:58 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:58.799+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:07:59.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:58 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:58.799+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:07:59.328 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:59 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:59.069+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:07:59.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:59 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:59.069+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:07:59.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:07:59 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:07:59.069+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:07:59.345 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:59 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:59.080+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:07:59.345 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:07:59 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:07:59.080+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:07:59.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:59 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:59.344+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:07:59.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:59 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:59.344+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:07:59.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:07:59 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:07:59.344+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:00.072 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:59 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:59.796+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:08:00.073 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:59 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:59.796+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.952408+0000 front 2024-04-28T13:07:28.952483+0000 (oldest deadline 2024-04-28T13:07:54.153760+0000) 2024-04-28T13:08:00.073 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:07:59 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:07:59.796+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:36.456218+0000 front 2024-04-28T13:07:36.455836+0000 (oldest deadline 2024-04-28T13:07:59.355798+0000) 2024-04-28T13:08:00.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:59 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:59.775+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:00.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:07:59 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:07:59.775+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:00.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:59 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:59.813+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:00.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:59 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:59.813+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:00.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:07:59 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:07:59.813+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:00.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:00 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:00.072+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:00.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:00 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:00.072+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:00.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:00 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:00.072+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:00.330 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:00 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:00.072+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:00.382 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:00 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:00.043+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:00.383 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:00 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:00.043+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:00.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:00 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:00.382+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:00.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:00 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:00.382+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:00.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:00 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:00.382+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:01.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:00 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:00.769+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:01.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:00 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:00.769+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:01.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:00 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:00.769+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:01.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:00 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:00.773+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:01.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:00 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:00.773+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:01.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:00 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:00.773+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:01.328 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:00 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:08:00.842+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:08:01.329 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:00 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:08:00.842+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.952408+0000 front 2024-04-28T13:07:28.952483+0000 (oldest deadline 2024-04-28T13:07:54.153760+0000) 2024-04-28T13:08:01.329 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:00 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:08:00.842+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:36.456218+0000 front 2024-04-28T13:07:36.455836+0000 (oldest deadline 2024-04-28T13:07:59.355798+0000) 2024-04-28T13:08:01.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:01.089+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:01.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:01.089+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:01.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:01.089+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:01.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:01.089+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:01.336 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:01 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:01.080+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:01.336 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:01 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:01.080+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:01.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:01 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:01.336+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:01.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:01 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:01.336+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:01.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:01 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:01.336+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:01.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:01 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:01.745+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:02.000 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:01 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:01.745+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:02.000 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:01 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:01.745+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:02.000 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:01 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:01.764+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:02.000 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:01 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:01.764+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:02.000 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:01 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:01.764+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:02.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:08:01.801+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:08:02.078 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:08:01.801+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.952408+0000 front 2024-04-28T13:07:28.952483+0000 (oldest deadline 2024-04-28T13:07:54.153760+0000) 2024-04-28T13:08:02.079 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:01 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:08:01.801+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:36.456218+0000 front 2024-04-28T13:07:36.455836+0000 (oldest deadline 2024-04-28T13:07:59.355798+0000) 2024-04-28T13:08:02.360 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:02 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:02.106+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:02.360 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:02 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:02.106+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:02.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:02.094+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:02.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:02.094+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:02.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:02.094+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:02.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:02.094+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:02.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:02 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:02.359+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:02.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:02 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:02.359+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:02.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:02 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:02.359+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:03.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:02 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:02.764+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:03.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:02 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:02.764+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:03.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:02 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:02.764+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:03.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:02 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:02.785+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:03.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:02 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:02.785+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:03.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:02 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:02.785+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:03.329 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:08:02.835+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.249411+0000 front 2024-04-28T13:07:20.249733+0000 (oldest deadline 2024-04-28T13:07:43.050484+0000) 2024-04-28T13:08:03.329 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:08:02.835+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.952408+0000 front 2024-04-28T13:07:28.952483+0000 (oldest deadline 2024-04-28T13:07:54.153760+0000) 2024-04-28T13:08:03.329 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:02 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6[56215]: 2024-04-28T13:08:02.835+0000 7fa85ce00700 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:36.456218+0000 front 2024-04-28T13:07:36.455836+0000 (oldest deadline 2024-04-28T13:07:59.355798+0000) 2024-04-28T13:08:03.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:03 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:03.076+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:03.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:03 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:03.076+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:03.330 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:03 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:03.076+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:03.330 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:03 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:03.076+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:03.404 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:03 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:03.080+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:03.404 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:03 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:03.080+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:03.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:03 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:03.404+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:03.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:03 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:03.404+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:03.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:03 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:03.404+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:04.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:03 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:03.752+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:04.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:03 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:03.752+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:04.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:03 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:03.752+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:04.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:03 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:03.784+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:04.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:03 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:03.784+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:04.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:03 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:03.784+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:04.304 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:04 smithi106 podman[86084]: 2024-04-28 13:08:04.000839829 +0000 UTC m=+5.646167238 container died 809fb8ebcacd8610af25b516614be7e98c620b72fa65af02915222c9fa876092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-28T13:08:04.305 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:04 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:04.027+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:04.305 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:04 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:04.027+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:04.305 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:04 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:04.027+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:04.305 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:04 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:04.027+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:04.386 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:04 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:04.097+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:04.386 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:04 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:04.097+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:04.578 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:04 smithi106 podman[86084]: 2024-04-28 13:08:04.399271024 +0000 UTC m=+6.044598436 container cleanup 809fb8ebcacd8610af25b516614be7e98c620b72fa65af02915222c9fa876092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:08:04.579 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:04 smithi106 bash[86084]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6 2024-04-28T13:08:04.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:04 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:04.386+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:04.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:04 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:04.386+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:04.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:04 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:04.386+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:05.067 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:04 smithi106 podman[86103]: 2024-04-28 13:08:04.692105401 +0000 UTC m=+0.676180477 container remove 809fb8ebcacd8610af25b516614be7e98c620b72fa65af02915222c9fa876092 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-28T13:08:05.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:04 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:04.779+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:05.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:04 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:04.779+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:05.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:04 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:04.779+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:05.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:04 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:04.799+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:05.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:04 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:04.799+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:05.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:04 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:04.799+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:05.328 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:05 smithi106 podman[86189]: 2024-04-28 13:08:05.23448918 +0000 UTC m=+0.114628636 container create a7a4bc4fe241df6163a3adf7b5ef33f79309a2ea35dbbd3eb5e19da0343cff00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=) 2024-04-28T13:08:05.329 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:05 smithi106 podman[86189]: 2024-04-28 13:08:05.156865874 +0000 UTC m=+0.037005300 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:08:05.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:05 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:05.067+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:05.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:05 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:05.067+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:05.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:05 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:05.067+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:05.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:05 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:05.067+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:05.402 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:05 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:05.087+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:05.402 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:05 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:05.087+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:05.403 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:05 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:05.087+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:34.537117+0000 front 2024-04-28T13:07:34.536978+0000 (oldest deadline 2024-04-28T13:08:04.537375+0000) 2024-04-28T13:08:05.660 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:05 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:05.416+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:05.660 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:05 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:05.416+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:05.660 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:05 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:05.416+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:05.660 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:05 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:05.416+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:05.828 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:05 smithi106 podman[86189]: 2024-04-28 13:08:05.47213441 +0000 UTC m=+0.352273876 container init a7a4bc4fe241df6163a3adf7b5ef33f79309a2ea35dbbd3eb5e19da0343cff00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424) 2024-04-28T13:08:05.829 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:05 smithi106 podman[86189]: 2024-04-28 13:08:05.484076227 +0000 UTC m=+0.364215657 container start a7a4bc4fe241df6163a3adf7b5ef33f79309a2ea35dbbd3eb5e19da0343cff00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS) 2024-04-28T13:08:05.829 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:05 smithi106 podman[86189]: 2024-04-28 13:08:05.570433805 +0000 UTC m=+0.450573270 container attach a7a4bc4fe241df6163a3adf7b5ef33f79309a2ea35dbbd3eb5e19da0343cff00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-28T13:08:06.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:05 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:05.789+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:06.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:05 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:05.789+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:06.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:05 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:05.789+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:06.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:05 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:05.811+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:06.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:05 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:05.811+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:06.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:05 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:05.811+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:06.313 INFO:journalctl@ceph.osd.6.smithi106.stdout:Apr 28 13:08:05 smithi106 podman[86189]: 2024-04-28 13:08:05.935129096 +0000 UTC m=+0.815268558 container died a7a4bc4fe241df6163a3adf7b5ef33f79309a2ea35dbbd3eb5e19da0343cff00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T13:08:06.313 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:06 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:06.053+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:06.313 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:06 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:06.053+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:06.313 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:06 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:06.053+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:06.313 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:06 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:06.053+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:06.407 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:06 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:06.102+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:06.407 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:06 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:06.102+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:06.407 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:06 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:06.102+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:34.537117+0000 front 2024-04-28T13:07:34.536978+0000 (oldest deadline 2024-04-28T13:08:04.537375+0000) 2024-04-28T13:08:06.669 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.6.service' 2024-04-28T13:08:06.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:06 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:06.406+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:06.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:06 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:06.406+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:06.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:06 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:06.406+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:06.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:06 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:06.406+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:06.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:08:06.774 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-28T13:08:06.774 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-28T13:08:06.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.7 2024-04-28T13:08:07.086 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:06 smithi129 systemd[1]: Stopping Ceph osd.7 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:08:07.086 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:07.085+0000 7f3f25f3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:08:07.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:06 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:06.803+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:07.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:06 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:06.803+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:07.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:06 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:06.803+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:07.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:06 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:06.825+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:07.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:06 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:06.825+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:07.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:06 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:06.825+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:07.396 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:07.085+0000 7f3f25f3c700 -1 osd.7 97 *** Got signal Terminated *** 2024-04-28T13:08:07.396 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:07.085+0000 7f3f25f3c700 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:08:07.396 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:07.098+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:07.396 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:07.098+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:07.396 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:07.098+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:34.537117+0000 front 2024-04-28T13:07:34.536978+0000 (oldest deadline 2024-04-28T13:08:04.537375+0000) 2024-04-28T13:08:07.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:07 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:07.082+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:07.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:07 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:07.082+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:07.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:07 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:07.082+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:07.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:07 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:07.082+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:07.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:07.396+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:07.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:07.396+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:07.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:07.396+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:07.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:07 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:07.396+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:08.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:07 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:07.808+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:08.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:07 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:07.808+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:08.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:07 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:07.808+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:08.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:07 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:07.872+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:08.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:07 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:07.872+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:08.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:07 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:07.872+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:08.417 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:08 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:08.147+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:08.417 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:08 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:08.147+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:08.417 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:08 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:08.147+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:34.537117+0000 front 2024-04-28T13:07:34.536978+0000 (oldest deadline 2024-04-28T13:08:04.537375+0000) 2024-04-28T13:08:08.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:08 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:08.078+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:08.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:08 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:08.078+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:08.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:08 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:08.078+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:08.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:08 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:08.078+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:08.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:08 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:08.417+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:08.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:08 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:08.417+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:08.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:08 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:08.417+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:08.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:08 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:08.417+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:08.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:08 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:08.417+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:09.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:08 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:08.828+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:09.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:08 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:08.828+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:09.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:08 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:08.828+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:09.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:08 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:08.876+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:09.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:08 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:08.876+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:09.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:08 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:08.876+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:09.428 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:09 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:09.166+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:09.429 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:09 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:09.166+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:09.429 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:09 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:09.166+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:34.537117+0000 front 2024-04-28T13:07:34.536978+0000 (oldest deadline 2024-04-28T13:08:04.537375+0000) 2024-04-28T13:08:09.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:09 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:09.115+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:09.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:09 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:09.115+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:09.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:09 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:09.115+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:09.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:09 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:09.115+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:09.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:09 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:09.429+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:09.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:09 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:09.429+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:09.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:09 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:09.429+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:09.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:09 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:09.429+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:09.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:09 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:09.429+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:10.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:09 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:09.815+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:10.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:09 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:09.815+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:10.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:09 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:09.815+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:10.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:09 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:09.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:10.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:09 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:09.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:10.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:09 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:09.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:10.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:09 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:09.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:10.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:10 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:10.070+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:10.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:10 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:10.070+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:10.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:10 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:10.070+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:10.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:10 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:10.070+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:10.462 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:10.209+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:10.462 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:10.209+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:10.462 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:10.209+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:34.537117+0000 front 2024-04-28T13:07:34.536978+0000 (oldest deadline 2024-04-28T13:08:04.537375+0000) 2024-04-28T13:08:10.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:10.461+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:10.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:10.461+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:10.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:10.461+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:10.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:10.461+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:10.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:10.461+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:11.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:10 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:10.857+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:11.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:10 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:10.857+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:11.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:10 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:10.857+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:11.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:10 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:10.859+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:11.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:10 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:10.859+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:11.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:10 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:10.859+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:11.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:10 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:10.859+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:11.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:11 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:11.078+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:11.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:11 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:11.078+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:11.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:11 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:11.078+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:11.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:11 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:11.078+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:11.517 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:11 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:11.256+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.734728+0000 front 2024-04-28T13:07:18.734698+0000 (oldest deadline 2024-04-28T13:07:44.034748+0000) 2024-04-28T13:08:11.518 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:11 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:11.256+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.535634+0000 front 2024-04-28T13:07:27.535887+0000 (oldest deadline 2024-04-28T13:07:51.035625+0000) 2024-04-28T13:08:11.518 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:11 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7[80499]: 2024-04-28T13:08:11.256+0000 7f3f22530700 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:34.537117+0000 front 2024-04-28T13:07:34.536978+0000 (oldest deadline 2024-04-28T13:08:04.537375+0000) 2024-04-28T13:08:11.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:11 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:11.437+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:11.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:11 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:11.437+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:11.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:11 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:11.437+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:11.519 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:11 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:11.437+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:11.519 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:11 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:11.437+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:12.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:11 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:11.853+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:12.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:11 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:11.853+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:12.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:11 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:11.853+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:12.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:11 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:11.853+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:41.494145+0000 front 2024-04-28T13:07:41.494048+0000 (oldest deadline 2024-04-28T13:08:10.894704+0000) 2024-04-28T13:08:12.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:11 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:11.843+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:12.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:11 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:11.843+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:12.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:11 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:11.843+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:12.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:11 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:11.843+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:12.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:12 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:12.117+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:12.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:12 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:12.117+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:12.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:12 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:12.117+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:12.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:12 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:12.117+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:12.695 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:12 smithi129 podman[97916]: 2024-04-28 13:08:12.411675751 +0000 UTC m=+5.456210671 container died 733a888307ad2fc9cd56908d3a0243f174fc585c7c72119667992629ce96cb43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-28T13:08:12.696 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:12 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:12.391+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:12.696 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:12 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:12.391+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:12.696 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:12 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:12.391+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:12.696 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:12 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:12.391+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:12.696 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:12 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:12.391+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:13.017 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:12 smithi129 podman[97916]: 2024-04-28 13:08:12.794848417 +0000 UTC m=+5.839383351 container cleanup 733a888307ad2fc9cd56908d3a0243f174fc585c7c72119667992629ce96cb43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-28T13:08:13.018 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:12 smithi129 bash[97916]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7 2024-04-28T13:08:13.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:12 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:12.871+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:13.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:12 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:12.871+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:13.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:12 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:12.871+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:13.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:12 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:12.871+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:41.494145+0000 front 2024-04-28T13:07:41.494048+0000 (oldest deadline 2024-04-28T13:08:10.894704+0000) 2024-04-28T13:08:13.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:12 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:12.873+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:13.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:12 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:12.873+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:13.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:12 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:12.873+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:13.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:12 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:12.873+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:13.406 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:13 smithi129 podman[97934]: 2024-04-28 13:08:13.046155094 +0000 UTC m=+0.633724584 container remove 733a888307ad2fc9cd56908d3a0243f174fc585c7c72119667992629ce96cb43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:08:13.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:13.143+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:13.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:13.144+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:13.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:13.144+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:13.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:13.144+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:13.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:13 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:13.144+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:13.740 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:13 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:13.438+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:13.740 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:13 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:13.438+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:13.741 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:13 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:13.438+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:13.741 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:13 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:13.438+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:13.741 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:13 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:13.438+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:13.741 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:13 smithi129 podman[98025]: 2024-04-28 13:08:13.310110419 +0000 UTC m=+0.018141709 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:08:13.741 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:13 smithi129 podman[98025]: 2024-04-28 13:08:13.430332852 +0000 UTC m=+0.138364142 container create 656e9356b31d4e62083a517e5345bf89c630e2da6ef80d3db39751a406b68004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-28T13:08:13.741 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:13 smithi129 podman[98025]: 2024-04-28 13:08:13.649266741 +0000 UTC m=+0.357298031 container init 656e9356b31d4e62083a517e5345bf89c630e2da6ef80d3db39751a406b68004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-28T13:08:13.741 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:13 smithi129 podman[98025]: 2024-04-28 13:08:13.652322868 +0000 UTC m=+0.360354166 container start 656e9356b31d4e62083a517e5345bf89c630e2da6ef80d3db39751a406b68004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-28T13:08:13.742 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:13 smithi129 podman[98025]: 2024-04-28 13:08:13.740918628 +0000 UTC m=+0.448949922 container attach 656e9356b31d4e62083a517e5345bf89c630e2da6ef80d3db39751a406b68004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-28T13:08:14.017 INFO:journalctl@ceph.osd.7.smithi129.stdout:Apr 28 13:08:13 smithi129 podman[98025]: 2024-04-28 13:08:13.944692479 +0000 UTC m=+0.652723765 container died 656e9356b31d4e62083a517e5345bf89c630e2da6ef80d3db39751a406b68004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:08:14.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:13 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:13.880+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:14.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:13 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:13.880+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:14.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:13 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:13.880+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:14.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:13 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:13.880+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:41.494145+0000 front 2024-04-28T13:07:41.494048+0000 (oldest deadline 2024-04-28T13:08:10.894704+0000) 2024-04-28T13:08:14.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:13 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:13.910+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:14.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:13 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:13.910+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:14.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:13 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:13.910+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:14.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:13 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:13.910+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:14.481 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:14 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:14.184+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:14.482 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:14 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:14.184+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:14.482 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:14 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:14.184+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:14.482 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:14 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:14.184+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:14.482 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:14 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:14.184+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:14.517 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:14 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:14.456+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:14.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:14 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:14.456+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:14.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:14 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:14.456+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:14.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:14 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:14.456+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:14.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:14 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:14.456+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:14.585 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.7.service' 2024-04-28T13:08:14.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:08:14.623 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-28T13:08:14.624 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-28T13:08:14.624 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.8 2024-04-28T13:08:14.900 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:14 smithi171 systemd[1]: Stopping Ceph osd.8 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:08:15.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:14.913+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:15.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:14.913+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:15.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:14.913+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:15.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:14.913+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:15.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:14.900+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:15.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:14.900+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:15.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:14.900+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:15.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:14.900+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:41.494145+0000 front 2024-04-28T13:07:41.494048+0000 (oldest deadline 2024-04-28T13:08:10.894704+0000) 2024-04-28T13:08:15.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:14.915+0000 7ff82f047700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:08:15.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:14.915+0000 7ff82f047700 -1 osd.8 97 *** Got signal Terminated *** 2024-04-28T13:08:15.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:14 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:14.915+0000 7ff82f047700 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:08:15.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:15 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:15.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:15.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:15 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:15.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:15.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:15 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:15.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:15.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:15 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:15.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:15.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:15 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:15.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:15.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:15 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:15.494+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:15.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:15 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:15.494+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:15.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:15 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:15.494+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:15.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:15 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:15.494+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:15.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:15 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:15.494+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:16.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:15 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:15.910+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:16.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:15 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:15.910+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:16.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:15 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:15.910+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:16.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:15 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:15.910+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:41.494145+0000 front 2024-04-28T13:07:41.494048+0000 (oldest deadline 2024-04-28T13:08:10.894704+0000) 2024-04-28T13:08:16.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:15 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:15.865+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:16.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:15 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:15.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:16.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:15 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:15.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:16.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:15 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:15.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:16.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:16 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:16.172+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:16.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:16 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:16.172+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:16.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:16 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:16.172+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:16.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:16 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:16.172+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:16.329 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:16 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:16.172+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:16.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:16 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:16.475+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:16.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:16 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:16.475+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:16.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:16 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:16.475+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:16.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:16 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:16.475+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:16.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:16 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:16.475+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:16.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:16 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:16.475+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:17.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:16.855+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:17.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:16.855+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:17.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:16.855+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:17.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:16.855+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:17.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:16.876+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:17.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:16.876+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:17.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:16.876+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:17.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:16.876+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:41.494145+0000 front 2024-04-28T13:07:41.494048+0000 (oldest deadline 2024-04-28T13:08:10.894704+0000) 2024-04-28T13:08:17.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:17 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:17.140+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:17.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:17 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:17.140+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:17.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:17 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:17.140+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:17.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:17 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:17.140+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:17.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:17 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:17.140+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:17.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:17 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:17.483+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:17.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:17 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:17.483+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:17.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:17 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:17.483+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:17.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:17 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:17.483+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:17.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:17 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:17.483+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:17.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:17 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:17.483+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:18.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:17 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:17.924+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:18.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:17 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:17.924+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:18.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:17 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:17.924+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:18.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:17 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:17.924+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:41.494145+0000 front 2024-04-28T13:07:41.494048+0000 (oldest deadline 2024-04-28T13:08:10.894704+0000) 2024-04-28T13:08:18.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:17 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:17.811+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:18.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:17 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:17.811+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:18.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:17 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:17.811+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:18.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:17 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:17.811+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:18.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:18 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:18.157+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:18.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:18 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:18.157+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:18.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:18 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:18.157+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:18.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:18 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:18.157+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:18.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:18 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:18.157+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:18.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:18 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:18.157+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:19.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:18 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:18.526+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:19.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:18 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:18.526+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:19.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:18 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:18.526+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:19.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:18 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:18.526+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:19.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:18 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:18.526+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:19.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:18 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:18.526+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:19.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:18.792+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:19.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:18.792+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:19.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:18.792+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:19.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:18.792+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:19.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:18.905+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:19.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:18.905+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:19.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:18.905+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:19.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:18.905+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:41.494145+0000 front 2024-04-28T13:07:41.494048+0000 (oldest deadline 2024-04-28T13:08:10.894704+0000) 2024-04-28T13:08:19.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:18 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:18.905+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:54.395455+0000 front 2024-04-28T13:07:54.395491+0000 (oldest deadline 2024-04-28T13:08:18.495656+0000) 2024-04-28T13:08:19.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:19.185+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:19.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:19.185+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:19.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:19.185+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:19.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:19.185+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:19.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:19.185+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:19.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:19 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:19.185+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:19.767 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:19 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:19.500+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:19.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:19 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:19.500+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:19.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:19 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:19.500+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:19.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:19 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:19.500+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:19.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:19 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:19.500+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:19.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:19 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:19.500+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:20.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:19.858+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:20.789865+0000 front 2024-04-28T13:07:20.789802+0000 (oldest deadline 2024-04-28T13:07:44.289894+0000) 2024-04-28T13:08:20.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:19.858+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:27.790822+0000 front 2024-04-28T13:07:27.790797+0000 (oldest deadline 2024-04-28T13:07:51.290937+0000) 2024-04-28T13:08:20.250 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:19.858+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:35.792974+0000 front 2024-04-28T13:07:35.792816+0000 (oldest deadline 2024-04-28T13:07:59.193125+0000) 2024-04-28T13:08:20.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:19.858+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:41.494145+0000 front 2024-04-28T13:07:41.494048+0000 (oldest deadline 2024-04-28T13:08:10.894704+0000) 2024-04-28T13:08:20.251 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8[81518]: 2024-04-28T13:08:19.858+0000 7ff82b63b700 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:54.395455+0000 front 2024-04-28T13:07:54.395491+0000 (oldest deadline 2024-04-28T13:08:18.495656+0000) 2024-04-28T13:08:20.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:19.813+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:20.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:19.813+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:20.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:19.813+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:20.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:19.813+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:20.252 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:19 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:19.813+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:20.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:20.230+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:20.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:20.230+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:20.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:20.230+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:20.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:20.230+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:20.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:20.230+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:20.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:20 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:20.230+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:20.625 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:20 smithi171 podman[98443]: 2024-04-28 13:08:20.328705797 +0000 UTC m=+5.514183857 container died eebdbc9b6b3359d79bf0e2d90d1cb612d6266a468733d6d791a8f80156f2217c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-28T13:08:20.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:20 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:20.508+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:20.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:20 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:20.508+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:20.768 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:20 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:20.508+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:20.769 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:20 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:20.508+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:20.769 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:20 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:20.508+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:20.769 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:20 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:20.508+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:20.951 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:20 smithi171 podman[98443]: 2024-04-28 13:08:20.73338193 +0000 UTC m=+5.918859991 container cleanup eebdbc9b6b3359d79bf0e2d90d1cb612d6266a468733d6d791a8f80156f2217c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-28T13:08:20.951 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:20 smithi171 bash[98443]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8 2024-04-28T13:08:20.952 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:20 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:20.781+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:20.952 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:20 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:20.781+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:20.952 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:20 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:20.781+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:20.952 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:20 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:20.781+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:20.952 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:20 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:20.781+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:21.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:20 smithi171 podman[98460]: 2024-04-28 13:08:20.951899226 +0000 UTC m=+0.622064763 container remove eebdbc9b6b3359d79bf0e2d90d1cb612d6266a468733d6d791a8f80156f2217c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-28T13:08:21.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:21 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:21.194+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:21.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:21 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:21.194+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:21.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:21 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:21.194+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:21.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:21 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:21.194+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:21.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:21 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:21.194+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:21.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:21 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:21.194+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:21.697 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:21 smithi171 podman[98554]: 2024-04-28 13:08:21.302085168 +0000 UTC m=+0.018085170 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:08:21.698 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:21 smithi171 podman[98554]: 2024-04-28 13:08:21.453268363 +0000 UTC m=+0.169268368 container create 27a44cc11d76112311dbc54d84830ff521a88dda859525041c4e140da0130308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-28T13:08:21.698 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:21 smithi171 podman[98554]: 2024-04-28 13:08:21.698009964 +0000 UTC m=+0.414009966 container init 27a44cc11d76112311dbc54d84830ff521a88dda859525041c4e140da0130308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-28T13:08:21.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:21 smithi171 podman[98554]: 2024-04-28 13:08:21.70137782 +0000 UTC m=+0.417377822 container start 27a44cc11d76112311dbc54d84830ff521a88dda859525041c4e140da0130308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-28T13:08:21.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:21 smithi171 podman[98554]: 2024-04-28 13:08:21.814759883 +0000 UTC m=+0.530759911 container attach 27a44cc11d76112311dbc54d84830ff521a88dda859525041c4e140da0130308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:08:21.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:21.786+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:21.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:21.786+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:21.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:21.786+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:21.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:21.786+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:21.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:21.786+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:22.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:21.540+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:22.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:21.540+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:22.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:21.540+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:22.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:21.540+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:22.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:21.540+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:22.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:21 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:21.540+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:22.249 INFO:journalctl@ceph.osd.8.smithi171.stdout:Apr 28 13:08:21 smithi171 podman[98554]: 2024-04-28 13:08:21.998098433 +0000 UTC m=+0.714098429 container died 27a44cc11d76112311dbc54d84830ff521a88dda859525041c4e140da0130308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-28T13:08:22.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:22 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:22.225+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:22.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:22 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:22.225+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:22.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:22 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:22.225+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:22.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:22 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:22.225+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:22.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:22 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:22.225+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:22.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:22 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:22.225+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:22.734 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.8.service' 2024-04-28T13:08:22.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:08:22.771 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-28T13:08:22.771 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-28T13:08:22.771 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.9 2024-04-28T13:08:22.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:22 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:22.789+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:22.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:22 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:22.789+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:23.000 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:22 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:22.789+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:23.000 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:22 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:22.789+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:23.000 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:22 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:22.789+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:23.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:22 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:22.570+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:23.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:22 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:22.570+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:23.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:22 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:22.570+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:23.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:22 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:22.570+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:23.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:22 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:22.570+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:23.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:22 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:22.570+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:23.216 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:22 smithi106 systemd[1]: Stopping Ceph osd.9 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:08:23.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:23.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:23.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:23.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:23.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:23.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:23.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:23.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:23.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:23.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:23.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:23.215+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:23.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:23.392+0000 7fb2c0600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:08:23.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:23.392+0000 7fb2c0600700 -1 osd.9 97 *** Got signal Terminated *** 2024-04-28T13:08:23.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:23 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:23.392+0000 7fb2c0600700 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:08:24.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:23.612+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:24.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:23.612+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:24.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:23.612+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:24.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:23.612+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:24.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:23.612+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:24.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:23 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:23.612+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:24.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:23 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:23.837+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:24.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:23 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:23.837+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:24.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:23 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:23.837+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:24.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:23 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:23.837+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:24.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:23 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:23.837+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:24.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:24 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:24.254+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:24.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:24 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:24.254+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:24.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:24 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:24.254+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:24.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:24 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:24.254+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:24.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:24 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:24.254+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:24.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:24 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:24.254+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:25.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:24.657+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:25.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:24.657+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:25.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:24.657+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:25.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:24.657+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:25.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:24.657+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:25.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:24 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:24.657+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:25.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:24 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:24.847+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:25.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:24 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:24.847+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:25.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:24 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:24.847+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:25.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:24 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:24.847+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:25.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:24 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:24.847+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:25.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:24 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:24.847+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:25.536 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:25 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:25.237+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:25.536 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:25 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:25.237+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:25.536 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:25 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:25.237+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:25.536 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:25 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:25.237+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:25.537 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:25 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:25.237+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:25.537 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:25 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:25.237+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:26.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:25 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:25.616+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:26.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:25 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:25.616+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:26.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:25 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:25.616+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:26.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:25 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:25.616+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:26.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:25 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:25.616+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:26.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:25 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:25.616+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:26.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:25 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:25.874+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:26.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:25 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:25.874+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:26.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:25 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:25.874+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:26.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:25 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:25.874+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:26.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:25 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:25.874+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:26.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:25 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:25.874+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:26.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:26 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:26.259+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:26.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:26 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:26.259+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:26.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:26 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:26.259+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:26.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:26 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:26.259+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:26.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:26 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:26.259+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:26.580 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:26 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:26.259+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:27.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:26 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:26.661+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:27.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:26 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:26.661+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:27.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:26 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:26.661+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:27.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:26 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:26.661+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:27.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:26 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:26.661+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:27.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:26 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:26.661+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:27.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:26 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:26.835+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:27.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:26 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:26.835+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:27.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:26 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:26.835+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:27.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:26 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:26.835+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:27.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:26 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:26.835+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:27.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:26 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:26.835+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:27.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:27 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:27.264+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:27.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:27 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:27.264+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:27.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:27 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:27.264+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:27.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:27 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:27.264+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:27.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:27 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:27.264+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:27.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:27 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:27.264+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:28.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:27 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:27.700+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:28.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:27 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:27.701+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:28.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:27 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:27.701+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:28.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:27 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:27.701+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:28.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:27 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:27.701+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:28.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:27 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:27.701+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:28.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:27 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:27.701+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:28.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:27 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:27.882+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:28.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:27 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:27.882+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:28.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:27 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:27.882+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:28.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:27 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:27.882+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:28.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:27 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:27.882+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:28.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:27 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:27.882+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:28.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:28.257+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.978208+0000 front 2024-04-28T13:07:09.978146+0000 (oldest deadline 2024-04-28T13:07:34.077677+0000) 2024-04-28T13:08:28.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:28.257+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:18.080636+0000 front 2024-04-28T13:07:18.080715+0000 (oldest deadline 2024-04-28T13:07:42.780797+0000) 2024-04-28T13:08:28.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:28.257+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:26.282805+0000 front 2024-04-28T13:07:26.282699+0000 (oldest deadline 2024-04-28T13:07:50.382903+0000) 2024-04-28T13:08:28.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:28.257+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.786248+0000 front 2024-04-28T13:07:33.785987+0000 (oldest deadline 2024-04-28T13:07:59.086236+0000) 2024-04-28T13:08:28.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:28.257+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:46.990905+0000 front 2024-04-28T13:07:46.991193+0000 (oldest deadline 2024-04-28T13:08:12.890791+0000) 2024-04-28T13:08:28.579 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:28 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9[62795]: 2024-04-28T13:08:28.257+0000 7fb2bbe00700 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.891757+0000 front 2024-04-28T13:07:52.891807+0000 (oldest deadline 2024-04-28T13:08:17.591780+0000) 2024-04-28T13:08:29.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:28 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:28.728+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:29.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:28 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:28.728+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:29.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:28 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:28.728+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:29.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:28 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:28.728+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:29.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:28 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:28.728+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:29.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:28 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:28.728+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:29.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:28 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:28.728+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:29.157 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:28 smithi106 podman[86306]: 2024-04-28 13:08:28.8852271 +0000 UTC m=+5.677069348 container died acb5a1a30184f1c874eb58e4b8aeba7c990a9759611c39fac596cd904d3eaab6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T13:08:29.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:28 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:28.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:29.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:28 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:28.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:29.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:28 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:28.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:29.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:28 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:28.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:29.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:28 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:28.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:29.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:28 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:28.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:29.437 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:29 smithi106 podman[86306]: 2024-04-28 13:08:29.268032123 +0000 UTC m=+6.059874401 container cleanup acb5a1a30184f1c874eb58e4b8aeba7c990a9759611c39fac596cd904d3eaab6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-28T13:08:29.437 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:29 smithi106 bash[86306]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9 2024-04-28T13:08:29.828 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:29 smithi106 podman[86324]: 2024-04-28 13:08:29.435646883 +0000 UTC m=+0.537763883 container remove acb5a1a30184f1c874eb58e4b8aeba7c990a9759611c39fac596cd904d3eaab6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:08:30.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:29 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:29.681+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:30.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:29 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:29.681+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:30.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:29 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:29.681+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:30.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:29 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:29.681+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:30.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:29 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:29.681+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:30.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:29 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:29.681+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:30.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:29 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:29.681+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:30.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:29 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:29.909+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:30.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:29 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:29.909+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:30.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:29 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:29.909+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:30.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:29 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:29.909+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:30.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:29 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:29.909+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:30.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:29 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:29.909+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:30.534 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:30 smithi106 podman[86410]: 2024-04-28 13:08:30.261467237 +0000 UTC m=+0.128967333 container create 5cf9c3049d817d792b4eba9628b76b6cecd1428f1aa3d6db0eaddc89d34f747e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:08:30.534 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:30 smithi106 podman[86410]: 2024-04-28 13:08:30.170233399 +0000 UTC m=+0.037733485 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:08:30.534 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:30 smithi106 podman[86410]: 2024-04-28 13:08:30.533646254 +0000 UTC m=+0.401146400 container init 5cf9c3049d817d792b4eba9628b76b6cecd1428f1aa3d6db0eaddc89d34f747e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-04-28T13:08:30.829 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:30 smithi106 podman[86410]: 2024-04-28 13:08:30.545172545 +0000 UTC m=+0.412672628 container start 5cf9c3049d817d792b4eba9628b76b6cecd1428f1aa3d6db0eaddc89d34f747e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True) 2024-04-28T13:08:30.829 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:30 smithi106 podman[86410]: 2024-04-28 13:08:30.62219936 +0000 UTC m=+0.489699529 container attach 5cf9c3049d817d792b4eba9628b76b6cecd1428f1aa3d6db0eaddc89d34f747e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T13:08:31.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:30.664+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:31.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:30.664+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:31.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:30.664+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:31.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:30.664+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:31.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:30.664+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:31.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:30.664+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:31.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:30 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:30.664+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:31.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:30 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:30.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:31.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:30 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:30.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:31.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:30 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:30.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:31.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:30 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:30.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:31.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:30 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:30.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:31.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:30 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:30.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:31.295 INFO:journalctl@ceph.osd.9.smithi106.stdout:Apr 28 13:08:31 smithi106 podman[86410]: 2024-04-28 13:08:31.005932114 +0000 UTC m=+0.873432221 container died 5cf9c3049d817d792b4eba9628b76b6cecd1428f1aa3d6db0eaddc89d34f747e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-28T13:08:31.665 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.9.service' 2024-04-28T13:08:31.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:08:31.765 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-28T13:08:31.765 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-28T13:08:31.765 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.10 2024-04-28T13:08:32.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:31 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:31.620+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:32.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:31 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:31.620+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:32.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:31 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:31.620+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:32.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:31 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:31.620+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:32.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:31 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:31.620+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:32.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:31 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:31.620+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:32.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:31 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:31.620+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:32.019 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:31 smithi129 systemd[1]: Stopping Ceph osd.10 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:08:32.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:31 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:31.946+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:32.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:31 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:31.946+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:32.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:31 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:31.946+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:32.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:31 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:31.946+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:32.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:31 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:31.946+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:32.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:31 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:31.946+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:32.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.088+0000 7f6fd1cd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:08:32.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.088+0000 7f6fd1cd0700 -1 osd.10 97 *** Got signal Terminated *** 2024-04-28T13:08:32.518 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.088+0000 7f6fd1cd0700 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:08:33.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.596+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:33.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.596+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:33.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.596+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:33.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.596+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:33.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.596+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:33.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.596+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:33.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:32 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:32.596+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:33.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:32 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:32.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:33.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:32 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:32.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:33.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:32 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:32.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:33.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:32 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:32.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:33.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:32 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:32.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:33.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:32 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:32.959+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:34.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:33 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:33.636+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:34.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:33 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:33.636+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:34.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:33 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:33.636+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:34.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:33 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:33.636+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:34.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:33 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:33.636+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:34.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:33 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:33.636+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:34.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:33 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:33.636+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:34.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:33 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:33.956+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:34.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:33 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:33.956+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:34.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:33 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:33.956+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:34.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:33 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:33.956+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:34.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:33 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:33.956+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:34.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:33 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:33.956+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:34.919 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:34 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:34.648+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:34.919 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:34 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:34.648+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:34.919 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:34 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:34.648+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:34.919 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:34 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:34.648+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:34.919 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:34 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:34.648+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:34.919 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:34 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:34.648+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:34.920 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:34 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:34.648+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:35.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:34 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:34.928+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:35.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:34 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:34.928+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:35.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:34 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:34.928+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:35.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:34 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:34.928+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:35.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:34 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:34.928+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:35.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:34 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:34.928+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:36.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:35 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:35.691+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:36.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:35 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:35.691+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:36.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:35 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:35.691+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:36.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:35 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:35.691+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:36.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:35 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:35.691+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:36.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:35 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:35.691+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:36.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:35 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:35.691+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:36.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:35 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:35.896+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:36.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:35 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:35.896+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:36.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:35 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:35.896+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:36.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:35 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:35.896+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:36.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:35 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:35.896+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:36.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:35 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:35.896+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:37.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:36 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:36.709+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:09.029961+0000 front 2024-04-28T13:07:09.029710+0000 (oldest deadline 2024-04-28T13:07:34.329667+0000) 2024-04-28T13:08:37.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:36 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:36.709+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6804 osd.1 since back 2024-04-28T13:07:21.231732+0000 front 2024-04-28T13:07:21.231785+0000 (oldest deadline 2024-04-28T13:07:45.931803+0000) 2024-04-28T13:08:37.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:36 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:36.709+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:28.832882+0000 front 2024-04-28T13:07:28.832833+0000 (oldest deadline 2024-04-28T13:07:52.332673+0000) 2024-04-28T13:08:37.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:36 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:36.709+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:33.433640+0000 front 2024-04-28T13:07:33.433738+0000 (oldest deadline 2024-04-28T13:08:04.633895+0000) 2024-04-28T13:08:37.018 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:36 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:36.709+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:44.634322+0000 front 2024-04-28T13:07:44.634360+0000 (oldest deadline 2024-04-28T13:08:08.134440+0000) 2024-04-28T13:08:37.019 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:36 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:36.709+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:52.835606+0000 front 2024-04-28T13:07:52.835516+0000 (oldest deadline 2024-04-28T13:08:15.735498+0000) 2024-04-28T13:08:37.019 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:36 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10[87395]: 2024-04-28T13:08:36.709+0000 7f6fce2c4700 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:08:03.237696+0000 front 2024-04-28T13:08:03.237600+0000 (oldest deadline 2024-04-28T13:08:27.337182+0000) 2024-04-28T13:08:37.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:36 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:36.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:37.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:36 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:36.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:37.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:36 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:36.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:37.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:36 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:36.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:37.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:36 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:36.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:37.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:36 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:36.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:37.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:36 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:36.912+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-04-28T13:08:10.269016+0000 front 2024-04-28T13:08:10.268894+0000 (oldest deadline 2024-04-28T13:08:36.168853+0000) 2024-04-28T13:08:37.715 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:37 smithi129 podman[98130]: 2024-04-28 13:08:37.447286833 +0000 UTC m=+5.484263944 container died 670606a19db608e84ed04aa55b2646f1971271cfbb9f3a3785517cc0ac8bfb7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-28T13:08:38.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:37 smithi129 podman[98130]: 2024-04-28 13:08:37.814337968 +0000 UTC m=+5.851315124 container cleanup 670606a19db608e84ed04aa55b2646f1971271cfbb9f3a3785517cc0ac8bfb7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T13:08:38.017 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:37 smithi129 bash[98130]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10 2024-04-28T13:08:38.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:37.899+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:38.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:37.899+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:38.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:37.899+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:38.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:37.899+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:38.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:37.899+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:38.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:37.899+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:38.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:37 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:37.899+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-04-28T13:08:10.269016+0000 front 2024-04-28T13:08:10.268894+0000 (oldest deadline 2024-04-28T13:08:36.168853+0000) 2024-04-28T13:08:38.438 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:38 smithi129 podman[98159]: 2024-04-28 13:08:38.082045485 +0000 UTC m=+0.634450873 container remove 670606a19db608e84ed04aa55b2646f1971271cfbb9f3a3785517cc0ac8bfb7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:08:38.710 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:38 smithi129 podman[98248]: 2024-04-28 13:08:38.342095223 +0000 UTC m=+0.018222558 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:08:38.710 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:38 smithi129 podman[98248]: 2024-04-28 13:08:38.440999526 +0000 UTC m=+0.117126836 container create 7641e31e0d417e1d144bb0250edb8cf805d5841247e57b2a3da0b137023c6882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-28T13:08:39.002 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:38 smithi129 podman[98248]: 2024-04-28 13:08:38.711193876 +0000 UTC m=+0.387321201 container init 7641e31e0d417e1d144bb0250edb8cf805d5841247e57b2a3da0b137023c6882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:08:39.003 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:38 smithi129 podman[98248]: 2024-04-28 13:08:38.714301336 +0000 UTC m=+0.390428644 container start 7641e31e0d417e1d144bb0250edb8cf805d5841247e57b2a3da0b137023c6882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T13:08:39.003 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:38 smithi129 podman[98248]: 2024-04-28 13:08:38.794532735 +0000 UTC m=+0.470660050 container attach 7641e31e0d417e1d144bb0250edb8cf805d5841247e57b2a3da0b137023c6882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:08:39.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:38.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:39.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:38.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:39.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:38.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:39.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:38.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:39.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:38.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:39.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:38.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:39.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:38 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:38.866+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-04-28T13:08:10.269016+0000 front 2024-04-28T13:08:10.268894+0000 (oldest deadline 2024-04-28T13:08:36.168853+0000) 2024-04-28T13:08:39.267 INFO:journalctl@ceph.osd.10.smithi129.stdout:Apr 28 13:08:39 smithi129 podman[98248]: 2024-04-28 13:08:39.003088599 +0000 UTC m=+0.679215924 container died 7641e31e0d417e1d144bb0250edb8cf805d5841247e57b2a3da0b137023c6882 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-28T13:08:39.616 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.10.service' 2024-04-28T13:08:39.654 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:08:39.654 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-28T13:08:39.654 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-28T13:08:39.654 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.11 2024-04-28T13:08:39.974 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 systemd[1]: Stopping Ceph osd.11 for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:08:39.974 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:39.974 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:39.974 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:39.974 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:39.974 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:39.975 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:39.975 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.908+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-04-28T13:08:10.269016+0000 front 2024-04-28T13:08:10.268894+0000 (oldest deadline 2024-04-28T13:08:36.168853+0000) 2024-04-28T13:08:39.975 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.973+0000 7fa18f72d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:08:39.975 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.973+0000 7fa18f72d700 -1 osd.11 97 *** Got signal Terminated *** 2024-04-28T13:08:39.975 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:39 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:39.973+0000 7fa18f72d700 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-28T13:08:41.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:40 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:40.888+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:41.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:40 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:40.888+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:41.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:40 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:40.888+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:41.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:40 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:40.888+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:41.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:40 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:40.888+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:41.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:40 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:40.888+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:41.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:40 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:40.888+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-04-28T13:08:10.269016+0000 front 2024-04-28T13:08:10.268894+0000 (oldest deadline 2024-04-28T13:08:36.168853+0000) 2024-04-28T13:08:42.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:41.848+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:42.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:41.848+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:42.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:41.848+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:42.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:41.848+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:42.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:41.848+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:42.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:41.848+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:42.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:41.848+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-04-28T13:08:10.269016+0000 front 2024-04-28T13:08:10.268894+0000 (oldest deadline 2024-04-28T13:08:36.168853+0000) 2024-04-28T13:08:42.251 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:41 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:41.848+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-28T13:08:16.169155+0000 front 2024-04-28T13:08:16.169191+0000 (oldest deadline 2024-04-28T13:08:41.469234+0000) 2024-04-28T13:08:43.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:42.857+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:43.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:42.857+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:43.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:42.857+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:43.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:42.857+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:43.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:42.857+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:43.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:42.857+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:43.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:42.857+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-04-28T13:08:10.269016+0000 front 2024-04-28T13:08:10.268894+0000 (oldest deadline 2024-04-28T13:08:36.168853+0000) 2024-04-28T13:08:43.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:42 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:42.857+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-28T13:08:16.169155+0000 front 2024-04-28T13:08:16.169191+0000 (oldest deadline 2024-04-28T13:08:41.469234+0000) 2024-04-28T13:08:44.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:43.865+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:44.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:43.865+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:44.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:43.865+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:44.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:43.865+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:44.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:43.865+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:44.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:43.865+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:44.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:43.865+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-04-28T13:08:10.269016+0000 front 2024-04-28T13:08:10.268894+0000 (oldest deadline 2024-04-28T13:08:36.168853+0000) 2024-04-28T13:08:44.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:43 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:43.865+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-28T13:08:16.169155+0000 front 2024-04-28T13:08:16.169191+0000 (oldest deadline 2024-04-28T13:08:41.469234+0000) 2024-04-28T13:08:45.249 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:44.834+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6806 osd.0 since back 2024-04-28T13:07:11.558876+0000 front 2024-04-28T13:07:11.558939+0000 (oldest deadline 2024-04-28T13:07:34.458731+0000) 2024-04-28T13:08:45.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:44.834+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-04-28T13:07:29.561806+0000 front 2024-04-28T13:07:29.561839+0000 (oldest deadline 2024-04-28T13:07:51.261936+0000) 2024-04-28T13:08:45.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:44.834+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6814 osd.3 since back 2024-04-28T13:07:38.263824+0000 front 2024-04-28T13:07:38.263749+0000 (oldest deadline 2024-04-28T13:07:59.963567+0000) 2024-04-28T13:08:45.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:44.834+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6812 osd.4 since back 2024-04-28T13:07:45.665808+0000 front 2024-04-28T13:07:45.665786+0000 (oldest deadline 2024-04-28T13:08:09.765732+0000) 2024-04-28T13:08:45.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:44.834+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-04-28T13:07:53.866527+0000 front 2024-04-28T13:07:53.866589+0000 (oldest deadline 2024-04-28T13:08:19.166681+0000) 2024-04-28T13:08:45.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:44.834+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.106:6822 osd.6 since back 2024-04-28T13:07:59.167446+0000 front 2024-04-28T13:07:59.167393+0000 (oldest deadline 2024-04-28T13:08:24.467134+0000) 2024-04-28T13:08:45.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:44.834+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6820 osd.7 since back 2024-04-28T13:08:10.269016+0000 front 2024-04-28T13:08:10.268894+0000 (oldest deadline 2024-04-28T13:08:36.168853+0000) 2024-04-28T13:08:45.250 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:44 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11[88099]: 2024-04-28T13:08:44.834+0000 7fa187d19700 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-04-28T13:08:16.169155+0000 front 2024-04-28T13:08:16.169191+0000 (oldest deadline 2024-04-28T13:08:41.469234+0000) 2024-04-28T13:08:45.557 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:45 smithi171 podman[98674]: 2024-04-28 13:08:45.286181063 +0000 UTC m=+5.433656709 container died f7364d4bcd90964b2ced812868eaa7fed82a8ebdf678e1122edde075f87304c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-28T13:08:45.908 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:45 smithi171 podman[98674]: 2024-04-28 13:08:45.649079146 +0000 UTC m=+5.796554791 container cleanup f7364d4bcd90964b2ced812868eaa7fed82a8ebdf678e1122edde075f87304c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:08:45.908 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:45 smithi171 bash[98674]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11 2024-04-28T13:08:46.243 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:45 smithi171 podman[98691]: 2024-04-28 13:08:45.908773126 +0000 UTC m=+0.618103170 container remove f7364d4bcd90964b2ced812868eaa7fed82a8ebdf678e1122edde075f87304c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-28T13:08:46.496 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:46 smithi171 podman[98781]: 2024-04-28 13:08:46.146536384 +0000 UTC m=+0.018320018 image pull 55dc54019ad82626637ea289d349113ba783f0c02a3b8251fba6bc77c1a60c07 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2 2024-04-28T13:08:46.496 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:46 smithi171 podman[98781]: 2024-04-28 13:08:46.284841281 +0000 UTC m=+0.156624948 container create fb4f0a346c739f237242854e06b20c7f249f08fc09b9a1c07ba7484b8c945b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T13:08:46.496 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:46 smithi171 podman[98781]: 2024-04-28 13:08:46.49654871 +0000 UTC m=+0.368332354 container init fb4f0a346c739f237242854e06b20c7f249f08fc09b9a1c07ba7484b8c945b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-28T13:08:46.749 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:46 smithi171 podman[98781]: 2024-04-28 13:08:46.500169871 +0000 UTC m=+0.371953496 container start fb4f0a346c739f237242854e06b20c7f249f08fc09b9a1c07ba7484b8c945b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T13:08:46.750 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:46 smithi171 podman[98781]: 2024-04-28 13:08:46.580137944 +0000 UTC m=+0.451921571 container attach fb4f0a346c739f237242854e06b20c7f249f08fc09b9a1c07ba7484b8c945b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-28T13:08:47.082 INFO:journalctl@ceph.osd.11.smithi171.stdout:Apr 28 13:08:46 smithi171 podman[98781]: 2024-04-28 13:08:46.793899802 +0000 UTC m=+0.665683431 container died fb4f0a346c739f237242854e06b20c7f249f08fc09b9a1c07ba7484b8c945b5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-28T13:08:47.379 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@osd.11.service' 2024-04-28T13:08:47.416 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:08:47.416 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-28T13:08:47.417 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-28T13:08:47.417 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.a 2024-04-28T13:08:47.828 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:08:47 smithi106 systemd[1]: Stopping Ceph mds.a for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:08:48.329 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:08:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a[69267]: 2024-04-28T13:08:47.976+0000 7f3d45000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:08:48.329 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:08:47 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a[69267]: 2024-04-28T13:08:47.976+0000 7f3d45000700 -1 mds.a *** got signal Terminated *** 2024-04-28T13:08:49.288 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:08:49 smithi106 podman[86527]: 2024-04-28 13:08:49.117988716 +0000 UTC m=+1.259736398 container died 03e72f101791c9d60f36560856413f2296f99b81685ead4386bd8edbcd843252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-28T13:08:49.829 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:08:49 smithi106 podman[86527]: 2024-04-28 13:08:49.560867026 +0000 UTC m=+1.702614745 container cleanup 03e72f101791c9d60f36560856413f2296f99b81685ead4386bd8edbcd843252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-28T13:08:49.829 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:08:49 smithi106 bash[86527]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a 2024-04-28T13:08:49.829 INFO:journalctl@ceph.mds.a.smithi106.stdout:Apr 28 13:08:49 smithi106 podman[86545]: 2024-04-28 13:08:49.736686593 +0000 UTC m=+0.612601097 container remove 03e72f101791c9d60f36560856413f2296f99b81685ead4386bd8edbcd843252 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-28T13:08:50.130 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.a.service' 2024-04-28T13:08:50.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:08:50.218 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-28T13:08:50.218 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-28T13:08:50.218 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.d 2024-04-28T13:08:50.578 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:08:50 smithi106 systemd[1]: Stopping Ceph mds.d for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:08:51.078 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:08:50 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d[68273]: 2024-04-28T13:08:50.695+0000 7fa152200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:08:51.079 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:08:50 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d[68273]: 2024-04-28T13:08:50.695+0000 7fa152200700 -1 mds.d *** got signal Terminated *** 2024-04-28T13:08:53.389 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:08:53 smithi106 podman[86660]: 2024-04-28 13:08:53.136173917 +0000 UTC m=+2.551000733 container died 058a2071450b79ce392eea528d6cb2b221358fb5f2e11686cd9d4ecbb76b8d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-28T13:08:53.716 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:08:53 smithi106 podman[86660]: 2024-04-28 13:08:53.506559671 +0000 UTC m=+2.921386487 container cleanup 058a2071450b79ce392eea528d6cb2b221358fb5f2e11686cd9d4ecbb76b8d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-28T13:08:53.716 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:08:53 smithi106 bash[86660]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d 2024-04-28T13:08:54.078 INFO:journalctl@ceph.mds.d.smithi106.stdout:Apr 28 13:08:53 smithi106 podman[86674]: 2024-04-28 13:08:53.71585129 +0000 UTC m=+0.562065368 container remove 058a2071450b79ce392eea528d6cb2b221358fb5f2e11686cd9d4ecbb76b8d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-28T13:08:54.259 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.d.service' 2024-04-28T13:08:54.362 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:08:54.362 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-28T13:08:54.363 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-28T13:08:54.363 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.g 2024-04-28T13:08:54.828 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:08:54 smithi106 systemd[1]: Stopping Ceph mds.g for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:08:55.329 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:08:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g[68909]: 2024-04-28T13:08:54.949+0000 7fb034000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:08:55.329 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:08:54 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g[68909]: 2024-04-28T13:08:54.949+0000 7fb034000700 -1 mds.g *** got signal Terminated *** 2024-04-28T13:08:57.448 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:08:57 smithi106 podman[86780]: 2024-04-28 13:08:57.139747982 +0000 UTC m=+2.316426975 container died 9adac8f9ea2c77ba618516f274ca94c684bc792819420723fc8c9069b0472ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:08:57.718 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:08:57 smithi106 podman[86780]: 2024-04-28 13:08:57.559883633 +0000 UTC m=+2.736562619 container cleanup 9adac8f9ea2c77ba618516f274ca94c684bc792819420723fc8c9069b0472ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-28T13:08:57.719 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:08:57 smithi106 bash[86780]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g 2024-04-28T13:08:58.079 INFO:journalctl@ceph.mds.g.smithi106.stdout:Apr 28 13:08:57 smithi106 podman[86794]: 2024-04-28 13:08:57.719028988 +0000 UTC m=+0.562721364 container remove 9adac8f9ea2c77ba618516f274ca94c684bc792819420723fc8c9069b0472ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-28T13:08:58.209 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.g.service' 2024-04-28T13:08:58.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:08:58.313 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-28T13:08:58.314 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-28T13:08:58.314 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.j 2024-04-28T13:08:58.578 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:08:58 smithi106 systemd[1]: Stopping Ceph mds.j for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:08:59.328 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:08:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j[67928]: 2024-04-28T13:08:58.861+0000 7f0e70000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:08:59.329 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:08:58 smithi106 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j[67928]: 2024-04-28T13:08:58.861+0000 7f0e70000700 -1 mds.j *** got signal Terminated *** 2024-04-28T13:09:01.409 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:09:01 smithi106 podman[86900]: 2024-04-28 13:09:01.127572571 +0000 UTC m=+2.453438773 container died 4f9290cb23b564e8a70e40ab24ba0c52d39aecaf3c62e2babdf9f1b73f883df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True) 2024-04-28T13:09:01.781 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:09:01 smithi106 podman[86900]: 2024-04-28 13:09:01.521884622 +0000 UTC m=+2.847750867 container cleanup 4f9290cb23b564e8a70e40ab24ba0c52d39aecaf3c62e2babdf9f1b73f883df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-28T13:09:01.781 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:09:01 smithi106 bash[86900]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j 2024-04-28T13:09:02.035 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:09:01 smithi106 podman[86914]: 2024-04-28 13:09:01.780979687 +0000 UTC m=+0.636023414 container remove 4f9290cb23b564e8a70e40ab24ba0c52d39aecaf3c62e2babdf9f1b73f883df9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-04-28T13:09:02.035 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:09:02 smithi106 systemd[1]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.j.service: Deactivated successfully. 2024-04-28T13:09:02.036 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:09:02 smithi106 systemd[1]: Stopped Ceph mds.j for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T13:09:02.036 INFO:journalctl@ceph.mds.j.smithi106.stdout:Apr 28 13:09:02 smithi106 systemd[1]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.j.service: Consumed 5.748s CPU time. 2024-04-28T13:09:02.038 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.j.service' 2024-04-28T13:09:02.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:09:02.144 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-28T13:09:02.145 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-28T13:09:02.145 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.b 2024-04-28T13:09:02.432 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:02 smithi129 systemd[1]: Stopping Ceph mds.b for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:09:02.767 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:02 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: 2024-04-28T13:09:02.431+0000 7f99b90e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:09:02.768 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:02 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b[92873]: 2024-04-28T13:09:02.431+0000 7f99b90e8700 -1 mds.b *** got signal Terminated *** 2024-04-28T13:09:05.407 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:05 smithi129 podman[98354]: 2024-04-28 13:09:05.067878282 +0000 UTC m=+2.735575771 container died 86d9c5e1c4dab9b8bf365cb8bde33604dfc3c585b13ed85a590c706da44d9408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:09:05.700 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:05 smithi129 podman[98354]: 2024-04-28 13:09:05.507196598 +0000 UTC m=+3.174894095 container cleanup 86d9c5e1c4dab9b8bf365cb8bde33604dfc3c585b13ed85a590c706da44d9408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-04-28T13:09:05.700 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:05 smithi129 bash[98354]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b 2024-04-28T13:09:05.935 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.b.service' 2024-04-28T13:09:05.955 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:05 smithi129 podman[98368]: 2024-04-28 13:09:05.699925324 +0000 UTC m=+0.633649410 container remove 86d9c5e1c4dab9b8bf365cb8bde33604dfc3c585b13ed85a590c706da44d9408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-28T13:09:05.955 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:05 smithi129 systemd[1]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.b.service: Deactivated successfully. 2024-04-28T13:09:05.955 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:05 smithi129 systemd[1]: Stopped Ceph mds.b for e5e35dfa-055b-11ef-bc93-c7b262605968. 2024-04-28T13:09:05.955 INFO:journalctl@ceph.mds.b.smithi129.stdout:Apr 28 13:09:05 smithi129 systemd[1]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.b.service: Consumed 7.101s CPU time. 2024-04-28T13:09:05.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:09:05.971 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-28T13:09:05.971 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-28T13:09:05.971 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.e 2024-04-28T13:09:06.267 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:09:06 smithi129 systemd[1]: Stopping Ceph mds.e for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:09:06.767 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:09:06 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e[92256]: 2024-04-28T13:09:06.293+0000 7f1e48772700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:09:06.767 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:09:06 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e[92256]: 2024-04-28T13:09:06.293+0000 7f1e48772700 -1 mds.e *** got signal Terminated *** 2024-04-28T13:09:09.353 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:09:09 smithi129 podman[98474]: 2024-04-28 13:09:09.059963668 +0000 UTC m=+2.868781199 container died 958082fb42cbf244b7490cb3166583c7ea6c035f30c2decb4cc7910302c9e275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:09:09.703 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:09:09 smithi129 podman[98474]: 2024-04-28 13:09:09.475435223 +0000 UTC m=+3.284252748 container cleanup 958082fb42cbf244b7490cb3166583c7ea6c035f30c2decb4cc7910302c9e275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-28T13:09:09.703 INFO:journalctl@ceph.mds.e.smithi129.stdout:Apr 28 13:09:09 smithi129 bash[98474]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-e 2024-04-28T13:09:09.857 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.e.service' 2024-04-28T13:09:09.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:09:09.891 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-28T13:09:09.891 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-28T13:09:09.891 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.h 2024-04-28T13:09:10.267 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:09:09 smithi129 systemd[1]: Stopping Ceph mds.h for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:09:10.268 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:09:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h[92566]: 2024-04-28T13:09:10.215+0000 7f848ace6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:09:10.268 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:09:10 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h[92566]: 2024-04-28T13:09:10.215+0000 7f848ace6700 -1 mds.h *** got signal Terminated *** 2024-04-28T13:09:13.332 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:09:13 smithi129 podman[98607]: 2024-04-28 13:09:13.05429562 +0000 UTC m=+2.947731266 container died 4b77b95139ba44ccf8f39ca0e629069d2df9b3ae1742a39f367513887e746ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-28T13:09:13.674 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:09:13 smithi129 podman[98607]: 2024-04-28 13:09:13.43207992 +0000 UTC m=+3.325515580 container cleanup 4b77b95139ba44ccf8f39ca0e629069d2df9b3ae1742a39f367513887e746ebb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:09:13.675 INFO:journalctl@ceph.mds.h.smithi129.stdout:Apr 28 13:09:13 smithi129 bash[98607]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-h 2024-04-28T13:09:13.813 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.h.service' 2024-04-28T13:09:13.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:09:13.845 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-28T13:09:13.845 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-28T13:09:13.845 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.k 2024-04-28T13:09:14.267 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 13:09:13 smithi129 systemd[1]: Stopping Ceph mds.k for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:09:14.268 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 13:09:14 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-k[91938]: 2024-04-28T13:09:14.177+0000 7eff52b0c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:09:14.268 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 13:09:14 smithi129 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-k[91938]: 2024-04-28T13:09:14.177+0000 7eff52b0c700 -1 mds.k *** got signal Terminated *** 2024-04-28T13:09:15.736 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 13:09:15 smithi129 podman[98732]: 2024-04-28 13:09:15.447573248 +0000 UTC m=+1.373040166 container died 3ebade331ac762b4877da2c2a267b85dc63ce670a99f9323ff77997ff00aa24d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:09:16.017 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 13:09:15 smithi129 podman[98732]: 2024-04-28 13:09:15.836060782 +0000 UTC m=+1.761527694 container cleanup 3ebade331ac762b4877da2c2a267b85dc63ce670a99f9323ff77997ff00aa24d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:09:16.017 INFO:journalctl@ceph.mds.k.smithi129.stdout:Apr 28 13:09:15 smithi129 bash[98732]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-k 2024-04-28T13:09:16.259 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.k.service' 2024-04-28T13:09:16.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:09:16.290 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-28T13:09:16.291 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-28T13:09:16.291 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.c 2024-04-28T13:09:16.669 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:09:16 smithi171 systemd[1]: Stopping Ceph mds.c for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:09:16.999 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:09:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c[93099]: 2024-04-28T13:09:16.669+0000 7f6715069700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:09:17.000 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:09:16 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c[93099]: 2024-04-28T13:09:16.669+0000 7f6715069700 -1 mds.c *** got signal Terminated *** 2024-04-28T13:09:20.999 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:09:20 smithi171 podman[98899]: 2024-04-28 13:09:20.677381188 +0000 UTC m=+4.190795844 container died d34769c4a5f88deca785c2378dae3ec6270d93151cd136cbef6524e425df7a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-28T13:09:21.337 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:09:21 smithi171 podman[98899]: 2024-04-28 13:09:21.086856575 +0000 UTC m=+4.600271231 container cleanup d34769c4a5f88deca785c2378dae3ec6270d93151cd136cbef6524e425df7a90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff) 2024-04-28T13:09:21.338 INFO:journalctl@ceph.mds.c.smithi171.stdout:Apr 28 13:09:21 smithi171 bash[98899]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-c 2024-04-28T13:09:21.461 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.c.service' 2024-04-28T13:09:21.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:09:21.496 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-28T13:09:21.496 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-28T13:09:21.496 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.f 2024-04-28T13:09:21.749 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:09:21 smithi171 systemd[1]: Stopping Ceph mds.f for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:09:22.249 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:09:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f[92789]: 2024-04-28T13:09:21.932+0000 7fc3160ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:09:22.249 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:09:21 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f[92789]: 2024-04-28T13:09:21.932+0000 7fc3160ec700 -1 mds.f *** got signal Terminated *** 2024-04-28T13:09:25.416 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:09:25 smithi171 podman[99022]: 2024-04-28 13:09:25.068471843 +0000 UTC m=+3.361526081 container died 82ae243700063fec78a8798001178ddcf1cf26fc1a77ed92aa8e5265d52f8ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-28T13:09:25.749 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:09:25 smithi171 podman[99022]: 2024-04-28 13:09:25.516447679 +0000 UTC m=+3.809501924 container cleanup 82ae243700063fec78a8798001178ddcf1cf26fc1a77ed92aa8e5265d52f8ab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-28T13:09:25.750 INFO:journalctl@ceph.mds.f.smithi171.stdout:Apr 28 13:09:25 smithi171 bash[99022]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-f 2024-04-28T13:09:25.936 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.f.service' 2024-04-28T13:09:25.967 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:09:25.967 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-28T13:09:25.968 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-28T13:09:25.968 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.i 2024-04-28T13:09:26.499 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:09:26 smithi171 systemd[1]: Stopping Ceph mds.i for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:09:26.500 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:09:26 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i[93407]: 2024-04-28T13:09:26.285+0000 7f52fa6f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:09:26.500 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:09:26 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i[93407]: 2024-04-28T13:09:26.285+0000 7f52fa6f9700 -1 mds.i *** got signal Terminated *** 2024-04-28T13:09:29.496 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:09:29 smithi171 podman[99143]: 2024-04-28 13:09:29.054401679 +0000 UTC m=+2.865185791 container died 56b0f9f78324d910e7b36b975b5c0888963f77a69d6e105511202d5539a31e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:09:29.750 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:09:29 smithi171 podman[99143]: 2024-04-28 13:09:29.579548957 +0000 UTC m=+3.390333068 container cleanup 56b0f9f78324d910e7b36b975b5c0888963f77a69d6e105511202d5539a31e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-28T13:09:29.750 INFO:journalctl@ceph.mds.i.smithi171.stdout:Apr 28 13:09:29 smithi171 bash[99143]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-i 2024-04-28T13:09:29.997 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.i.service' 2024-04-28T13:09:30.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:09:30.028 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-28T13:09:30.028 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-28T13:09:30.028 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.l 2024-04-28T13:09:30.499 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 13:09:30 smithi171 systemd[1]: Stopping Ceph mds.l for e5e35dfa-055b-11ef-bc93-c7b262605968... 2024-04-28T13:09:30.500 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 13:09:30 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l[92479]: 2024-04-28T13:09:30.333+0000 7f48c8752700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-28T13:09:30.500 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 13:09:30 smithi171 ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l[92479]: 2024-04-28T13:09:30.333+0000 7f48c8752700 -1 mds.l *** got signal Terminated *** 2024-04-28T13:09:32.742 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 13:09:32 smithi171 podman[99263]: 2024-04-28 13:09:32.445343774 +0000 UTC m=+2.206039975 container died 758fa61b1c1eef28a963fd68ea50093aebcdd115b52cb7a045888b233c23bd9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-28T13:09:32.743 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 13:09:32 smithi171 podman[99263]: 2024-04-28 13:09:32.742949207 +0000 UTC m=+2.503645397 container cleanup 758fa61b1c1eef28a963fd68ea50093aebcdd115b52cb7a045888b233c23bd9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-28T13:09:32.999 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 13:09:32 smithi171 bash[99263]: ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l 2024-04-28T13:09:33.000 INFO:journalctl@ceph.mds.l.smithi171.stdout:Apr 28 13:09:32 smithi171 podman[99277]: 2024-04-28 13:09:32.95342003 +0000 UTC m=+0.511522487 container remove 758fa61b1c1eef28a963fd68ea50093aebcdd115b52cb7a045888b233c23bd9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:39d00e115f9430aba1116fce52d7d386b73fc3b9ed873e7f9c2c3b6cdcd253c2, name=ceph-e5e35dfa-055b-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240426.111959-bc612ff, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-28T13:09:33.089 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e5e35dfa-055b-11ef-bc93-c7b262605968@mds.l.service' 2024-04-28T13:09:33.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-28T13:09:33.120 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-28T13:09:33.120 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-28T13:09:33.435 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T13:09:38.360 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-28T13:09:38.550 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T13:09:40.904 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-28T13:09:41.095 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T13:09:43.408 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-28T13:09:43.500 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-28T13:09:43.532 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-28T13:09:43.557 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-28T13:09:43.559 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/remote/smithi106/crash 2024-04-28T13:09:43.560 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/crash -- . 2024-04-28T13:09:43.652 INFO:teuthology.orchestra.run.smithi106.stderr:tar: /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-28T13:09:43.652 INFO:teuthology.orchestra.run.smithi106.stderr:tar: Error is not recoverable: exiting now 2024-04-28T13:09:43.657 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/remote/smithi129/crash 2024-04-28T13:09:43.658 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/crash -- . 2024-04-28T13:09:43.701 INFO:teuthology.orchestra.run.smithi129.stderr:tar: /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-28T13:09:43.701 INFO:teuthology.orchestra.run.smithi129.stderr:tar: Error is not recoverable: exiting now 2024-04-28T13:09:43.702 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/remote/smithi171/crash 2024-04-28T13:09:43.704 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/crash -- . 2024-04-28T13:09:43.768 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-28T13:09:43.768 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-04-28T13:09:43.769 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-28T13:09:43.770 DEBUG:teuthology.orchestra.run.smithi106:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-28T13:09:43.878 INFO:tasks.cephadm:Compressing logs... 2024-04-28T13:09:43.879 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-28T13:09:43.883 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-28T13:09:43.886 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-28T13:09:43.908 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log 2024-04-28T13:09:43.908 INFO:teuthology.orchestra.run.smithi171.stderr:: No such file or directory 2024-04-28T13:09:43.908 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-28T13:09:43.910 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-28T13:09:43.910 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-28T13:09:43.910 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-28T13:09:43.910 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-28T13:09:43.911 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-28T13:09:43.912 INFO:teuthology.orchestra.run.smithi129.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-04-28T13:09:43.912 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log 2024-04-28T13:09:43.912 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-28T13:09:43.912 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-28T13:09:43.913 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-28T13:09:43.918 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-28T13:09:43.918 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mon.c.log: 88.9% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-28T13:09:43.918 INFO:teuthology.orchestra.run.smithi171.stderr: 90.4% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-28T13:09:43.919 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-28T13:09:43.919 INFO:teuthology.orchestra.run.smithi171.stderr:gzip/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mgr.z.log: -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-28T13:09:43.919 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-28T13:09:43.919 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-28T13:09:43.919 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-28T13:09:43.919 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-28T13:09:43.919 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-28T13:09:43.919 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-28T13:09:43.920 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-28T13:09:43.920 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-28T13:09:43.920 INFO:teuthology.orchestra.run.smithi129.stderr: 88.9% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-28T13:09:43.920 INFO:teuthology.orchestra.run.smithi129.stderr: 90.4%/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.h.log: -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-28T13:09:43.920 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-28T13:09:43.920 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.10.log: 93.1% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-28T13:09:43.921 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.11.log: 93.2% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-28T13:09:43.928 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-28T13:09:43.928 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-28T13:09:43.928 INFO:teuthology.orchestra.run.smithi129.stderr: 82.4% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-28T13:09:43.928 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-28T13:09:43.929 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi106.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log 2024-04-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-28T13:09:43.939 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-28T13:09:43.940 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-28T13:09:43.940 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-28T13:09:43.941 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-28T13:09:43.944 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.0.log 90.3% 2024-04-28T13:09:43.944 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-28T13:09:43.944 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-28T13:09:43.945 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-28T13:09:43.946 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-28T13:09:43.947 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-28T13:09:43.949 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-28T13:09:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.6.log: /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-28T13:09:43.951 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-28T13:09:43.951 INFO:teuthology.orchestra.run.smithi106.stderr: 82.7% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-28T13:09:43.953 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-28T13:09:43.957 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-volume.log: --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-28T13:09:43.959 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-28T13:09:43.960 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-28T13:09:43.962 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-28T13:09:43.965 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-28T13:09:43.966 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-28T13:09:43.967 INFO:teuthology.orchestra.run.smithi171.stderr: 94.1% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-28T13:09:43.969 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-28T13:09:43.971 INFO:teuthology.orchestra.run.smithi171.stderr: 94.1% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-28T13:09:43.973 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-28T13:09:43.975 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-28T13:09:43.982 INFO:teuthology.orchestra.run.smithi106.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-28T13:09:43.987 INFO:teuthology.orchestra.run.smithi171.stderr: 94.8% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-28T13:09:43.991 INFO:teuthology.orchestra.run.smithi106.stderr: 90.9% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-28T13:09:44.002 INFO:teuthology.orchestra.run.smithi129.stderr: 94.5% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-28T13:09:44.013 INFO:teuthology.orchestra.run.smithi129.stderr: 94.6% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-28T13:09:44.027 INFO:teuthology.orchestra.run.smithi106.stderr: 94.5% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-28T13:09:44.027 INFO:teuthology.orchestra.run.smithi106.stderr: 93.9% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-28T13:09:44.028 INFO:teuthology.orchestra.run.smithi106.stderr: 93.8% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-28T13:09:44.076 INFO:teuthology.orchestra.run.smithi106.stderr: 94.5% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-28T13:09:44.133 INFO:teuthology.orchestra.run.smithi129.stderr: 93.9% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-28T13:09:44.305 INFO:teuthology.orchestra.run.smithi171.stderr: 91.9% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-28T13:09:44.334 INFO:teuthology.orchestra.run.smithi129.stderr: 92.0% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-28T13:09:44.336 INFO:teuthology.orchestra.run.smithi106.stderr: 90.1% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-28T13:09:44.761 INFO:teuthology.orchestra.run.smithi171.stderr: 92.5% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-28T13:09:44.912 INFO:teuthology.orchestra.run.smithi129.stderr: 92.4% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-28T13:09:44.945 INFO:teuthology.orchestra.run.smithi106.stderr: 92.1% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-28T13:09:44.951 INFO:teuthology.orchestra.run.smithi171.stderr: 92.4% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-28T13:09:44.981 INFO:teuthology.orchestra.run.smithi129.stderr: 92.5% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-28T13:09:45.001 INFO:teuthology.orchestra.run.smithi106.stderr: 90.4% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-28T13:09:45.035 INFO:teuthology.orchestra.run.smithi171.stderr: 93.6% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-28T13:09:45.080 INFO:teuthology.orchestra.run.smithi106.stderr: 92.7% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-28T13:09:45.150 INFO:teuthology.orchestra.run.smithi106.stderr: 92.1% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-28T13:09:45.229 INFO:teuthology.orchestra.run.smithi129.stderr: 93.2% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-28T13:09:45.270 INFO:teuthology.orchestra.run.smithi106.stderr: 93.3% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-28T13:09:45.276 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-28T13:09:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m1.362s 2024-04-28T13:09:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m5.836s 2024-04-28T13:09:45.276 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m0.562s 2024-04-28T13:09:45.386 INFO:teuthology.orchestra.run.smithi129.stderr: 93.2% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-28T13:09:45.388 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-04-28T13:09:45.388 INFO:teuthology.orchestra.run.smithi129.stderr:real 0m1.488s 2024-04-28T13:09:45.388 INFO:teuthology.orchestra.run.smithi129.stderr:user 0m5.472s 2024-04-28T13:09:45.389 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m0.213s 2024-04-28T13:09:45.435 INFO:teuthology.orchestra.run.smithi171.stderr: 93.1% -- replaced with /var/log/ceph/e5e35dfa-055b-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-04-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi171.stderr:real 0m1.537s 2024-04-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi171.stderr:user 0m4.949s 2024-04-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m0.191s 2024-04-28T13:09:45.437 INFO:tasks.cephadm:Archiving logs... 2024-04-28T13:09:45.438 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/remote/smithi106/log 2024-04-28T13:09:45.439 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-28T13:09:46.329 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/remote/smithi129/log 2024-04-28T13:09:46.330 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-28T13:09:47.142 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/remote/smithi171/log 2024-04-28T13:09:47.143 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-28T13:09:47.927 INFO:tasks.cephadm:Removing cluster... 2024-04-28T13:09:47.928 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 --force 2024-04-28T13:09:48.253 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T13:09:48.922 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 --force 2024-04-28T13:09:49.114 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T13:09:49.370 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e5e35dfa-055b-11ef-bc93-c7b262605968 --force 2024-04-28T13:09:49.559 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: e5e35dfa-055b-11ef-bc93-c7b262605968 2024-04-28T13:09:49.814 INFO:tasks.cephadm:Removing cephadm ... 2024-04-28T13:09:49.814 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-28T13:09:49.875 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-28T13:09:49.891 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-28T13:09:49.906 INFO:tasks.cephadm:Teardown complete 2024-04-28T13:09:49.907 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-28T13:09:49.917 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-28T13:09:49.917 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-28T13:09:49.925 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-28T13:09:49.933 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-28T13:09:50.074 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-28T13:09:50.083 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-28T13:09:50.083 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T13:09:50.091 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T13:09:50.093 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-28T13:09:50.105 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-04-28T13:09:50.107 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-04-28T13:09:50.109 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T13:09:50.109 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-04-28T13:09:50.110 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +1906us[+1859us] +/- 52ms 2024-04-28T13:09:50.110 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 -1021us[-1021us] +/- 71ms 2024-04-28T13:09:50.110 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -1546us[-1546us] +/- 64ms 2024-04-28T13:09:50.110 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:09:50.111 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T13:09:50.112 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-04-28T13:09:50.112 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 +1816us[+1816us] +/- 52ms 2024-04-28T13:09:50.112 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -1103us[-1103us] +/- 71ms 2024-04-28T13:09:50.112 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 -1629us[-1629us] +/- 64ms 2024-04-28T13:09:50.112 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:09:50.139 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-28T13:09:50.149 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-28T13:09:50.149 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-28T13:09:50.150 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 +1876us[+1898us] +/- 52ms 2024-04-28T13:09:50.150 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 -1049us[-1049us] +/- 71ms 2024-04-28T13:09:50.150 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -1586us[-1586us] +/- 64ms 2024-04-28T13:09:50.150 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-28T13:09:50.151 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-28T13:09:50.161 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-28T13:09:50.162 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-28T13:09:50.188 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-28T13:09:50.371 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-28T13:09:50.434 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-28T13:09:50.498 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T13:09:50.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:09:50.586 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T13:09:50.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:09:50.613 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-28T13:09:50.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:09:50.640 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-28T13:09:50.650 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-28T13:09:50.679 INFO:teuthology.task.internal:Duration was 2610.867618 seconds 2024-04-28T13:09:50.680 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-28T13:09:50.730 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-28T13:09:50.730 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-28T13:09:50.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-28T13:09:50.740 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-28T13:09:50.774 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T13:09:50.776 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T13:09:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-28T13:09:51.424 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-28T13:09:51.424 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-04-28T13:09:51.424 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-28T13:09:51.492 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-04-28T13:09:51.493 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-28T13:09:51.515 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-04-28T13:09:51.515 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-28T13:09:51.564 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-28T13:09:51.564 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-28T13:09:51.572 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-28T13:09:51.574 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-28T13:09:51.653 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-28T13:09:51.654 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-28T13:09:51.659 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-28T13:09:51.661 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-28T13:09:52.132 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-28T13:09:52.144 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-28T13:09:52.144 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-28T13:09:52.230 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-28T13:09:52.254 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-28T13:09:52.278 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-28T13:09:52.289 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:09:52.297 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:09:52.299 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:09:52.318 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-04-28T13:09:52.340 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-04-28T13:09:52.388 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-04-28T13:09:52.432 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:09:52.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:09:52.479 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:09:52.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:09:52.494 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-28T13:09:52.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-28T13:09:52.508 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-28T13:09:52.518 INFO:teuthology.task.internal:Transferring archived files... 2024-04-28T13:09:52.519 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/remote/smithi106 2024-04-28T13:09:52.519 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-28T13:09:52.622 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/remote/smithi129 2024-04-28T13:09:52.623 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-28T13:09:52.657 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882/remote/smithi171 2024-04-28T13:09:52.657 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-28T13:09:52.691 INFO:teuthology.task.internal:Removing archive directory... 2024-04-28T13:09:52.692 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-28T13:09:52.700 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-28T13:09:52.702 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-28T13:09:52.750 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-28T13:09:52.763 INFO:teuthology.task.internal:Not uploading archives. 2024-04-28T13:09:52.764 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-28T13:09:52.789 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-28T13:09:52.790 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-28T13:09:52.797 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-28T13:09:52.799 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-28T13:09:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 28 13:09 /home/ubuntu/cephtest 2024-04-28T13:09:52.814 INFO:teuthology.orchestra.run.smithi171.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 28 13:09 /home/ubuntu/cephtest 2024-04-28T13:09:52.847 INFO:teuthology.orchestra.run.smithi106.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 28 13:09 /home/ubuntu/cephtest 2024-04-28T13:09:52.850 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-28T13:09:52.861 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-28T13:09:52.939 INFO:teuthology.nuke:Checking targets against current locks 2024-04-28T13:09:52.965 DEBUG:teuthology.nuke:shortname: smithi106 2024-04-28T13:09:52.965 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T13:09:52.991 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:16.228162', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEIkHMXbbD8+hFeko3auXYSTn1U+R4Jvg6s6D1VvWpNpIaWZ3ipFyVXRkbetjB3dx0/6Nz3XztJamto3A3rHySY='} 2024-04-28T13:09:52.995 DEBUG:teuthology.nuke:shortname: smithi129 2024-04-28T13:09:52.995 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T13:09:53.017 DEBUG:teuthology.nuke:shortname: smithi171 2024-04-28T13:09:53.017 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-28T13:09:53.021 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:16.226939', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDsPJ7LPnuQZtmVlRbuR5Cq3lxEvksEO95xJc/qlvWrk3lVwrU5/15o0OjhOw+izl6mc2fkmYTXh0gEWCGcnDLE='} 2024-04-28T13:09:53.051 INFO:teuthology.orchestra.console.smithi106:Power off 2024-04-28T13:09:53.052 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-28T13:09:53.066 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-28_11:41:23-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7676882', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-28 12:12:16.229271', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKkE+47+lLqX/2/2xhe2CM41Z7iwkeNOx2ZAE8YxafIUhVkPqow7M0DqIbZcEkzbI1Ze0Kis5IN6VdlHuclbd+0='} 2024-04-28T13:09:53.075 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2024-04-28T13:09:53.075 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:09:53.100 INFO:teuthology.orchestra.console.smithi129:Power off 2024-04-28T13:09:53.100 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-28T13:09:53.114 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-28T13:09:53.125 DEBUG:teuthology.orchestra.console.smithi129:power off output: Chassis Power Control: Down/Off 2024-04-28T13:09:53.125 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:09:53.144 INFO:teuthology.orchestra.console.smithi171:Power off 2024-04-28T13:09:53.144 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-28T13:09:53.154 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2024-04-28T13:09:53.167 DEBUG:teuthology.orchestra.console.smithi171:power off output: Chassis Power Control: Down/Off 2024-04-28T13:09:53.168 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:09:53.196 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-04-28T13:09:57.116 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:09:57.155 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:09:57.197 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:09:57.226 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-28T13:09:57.265 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2024-04-28T13:09:57.307 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is on 2024-04-28T13:10:01.229 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:10:01.266 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:10:01.307 DEBUG:teuthology.orchestra.console.smithi171:pexpect command: ipmitool -H smithi171.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:10:01.338 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-28T13:10:01.375 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is on 2024-04-28T13:10:01.417 DEBUG:teuthology.orchestra.console.smithi171:check power output: Chassis Power is off 2024-04-28T13:10:01.518 INFO:teuthology.orchestra.console.smithi171:Power off completed 2024-04-28T13:10:05.341 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:10:05.376 DEBUG:teuthology.orchestra.console.smithi129:pexpect command: ipmitool -H smithi129.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-28T13:10:05.451 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-04-28T13:10:05.487 DEBUG:teuthology.orchestra.console.smithi129:check power output: Chassis Power is off 2024-04-28T13:10:05.553 INFO:teuthology.orchestra.console.smithi106:Power off completed 2024-04-28T13:10:05.588 INFO:teuthology.orchestra.console.smithi129:Power off completed 2024-04-28T13:10:05.694 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/suites/iogen}} duration: 2610.867618083954 failure_reason: 'Command failed on smithi106 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c ''ceph fs subvolumegroup pin cephfs qa random 0.10''"' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1888f5ee7d9b4e26a97acb041ba7abf9 status: fail success: false 2024-04-28T13:10:05.694 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-28T13:10:05.776 INFO:teuthology.run:FAIL